Artificial Intelligence in Finance: A Python-Based Guide [1 ed.] 1492055433, 9781492055433

Citation preview

Artificial Intelligence in Finance A Python-Based Guide

Yves Hilpisch

Artificial Intelligence in Finance A Python-Based Guide

Yves Hilpisch

Beijing

Boston Farnham Sebastopol

Tokyo

Artificial Intelligence in Finance by Yves Hilpisch Copyright © 2021 Yves Hilpisch. All rights reserved. Printed in the United States of America. Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or [email protected].

Acquisitions Editor: Michelle Smith Development Editor: Corbin Collins Production Editor: Daniel Elfanbaum Copyeditor: Piper Editorial, LLC Proofreader: JM Olejarz October 2020:

Indexer: Potomac Indexing, LLC Interior Designer: David Futato Cover Designer: Karen Montgomery Illustrator: O’Reilly Media, Inc.

First Edition

Revision History for the First Edition 2020-10-14:

First Release

See http://oreilly.com/catalog/errata.csp?isbn=9781492055433 for release details. The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Artificial Intelligence in Finance, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc. The views expressed in this work are those of the author, and do not represent the publisher’s views. While the publisher and the author have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. This book is not intended as financial advice. Please consult a qualified professional if you require financial advice.

978-1-492-05543-3 [LSI]

Table of Contents

Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

Part I.

Machine Intelligence

1. Artificial Intelligence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Algorithms Types of Data Types of Learning Types of Tasks Types of Approaches Neural Networks OLS Regression Estimation with Neural Networks Classification with Neural Networks Importance of Data Small Data Set Larger Data Set Big Data Conclusions References

3 4 4 8 8 9 9 13 20 22 23 26 28 29 30

2. Superintelligence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Success Stories Atari Go Chess Importance of Hardware

32 32 38 40 42 iii

Forms of Intelligence Paths to Superintelligence Networks and Organizations Biological Enhancements Brain-Machine Hybrids Whole Brain Emulation Artificial Intelligence Intelligence Explosion Goals and Control Superintelligence and Goals Superintelligence and Control Potential Outcomes Conclusions References

44 45 46 46 47 48 49 50 50 51 53 54 56 56

Part II. Finance and Machine Learning 3. Normative Finance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Uncertainty and Risk Definitions Numerical Example Expected Utility Theory Assumptions and Results Numerical Example Mean-Variance Portfolio Theory Assumptions and Results Numerical Example Capital Asset Pricing Model Assumptions and Results Numerical Example Arbitrage Pricing Theory Assumptions and Results Numerical Example Conclusions References

62 62 63 66 66 69 72 72 75 82 83 85 90 91 93 95 96

4. Data-Driven Finance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Scientific Method Financial Econometrics and Regression Data Availability Programmatic APIs

iv

|

Table of Contents

100 101 104 105

Structured Historical Data Structured Streaming Data Unstructured Historical Data Unstructured Streaming Data Alternative Data Normative Theories Revisited Expected Utility and Reality Mean-Variance Portfolio Theory Capital Asset Pricing Model Arbitrage Pricing Theory Debunking Central Assumptions Normally Distributed Returns Linear Relationships Conclusions References Python Code

105 108 110 112 113 117 118 123 130 134 143 143 153 155 156 156

5. Machine Learning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161 Learning Data Success Capacity Evaluation Bias and Variance Cross-Validation Conclusions References

162 162 165 169 172 178 180 183 183

6. AI-First Finance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185 Efficient Markets Market Prediction Based on Returns Data Market Prediction with More Features Market Prediction Intraday Conclusions References

Part III.

186 192 199 204 205 207

Statistical Inefficiencies

7. Dense Neural Networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 The Data Baseline Prediction

212 214

Table of Contents

|

v

Normalization Dropout Regularization Bagging Optimizers Conclusions References

218 220 222 225 227 228 228

8. Recurrent Neural Networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 First Example Second Example Financial Price Series Financial Return Series Financial Features Estimation Classification Deep RNNs Conclusions References

230 234 237 240 242 243 244 245 246 247

9. Reinforcement Learning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 Fundamental Notions OpenAI Gym Monte Carlo Agent Neural Network Agent DQL Agent Simple Finance Gym Better Finance Gym FQL Agent Conclusions References

250 251 255 257 260 264 268 271 277 278

Part IV. Algorithmic Trading 10. Vectorized Backtesting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 Backtesting an SMA-Based Strategy Backtesting a Daily DNN-Based Strategy Backtesting an Intraday DNN-Based Strategy Conclusions References

vi

|

Table of Contents

282 289 295 301 301

11. Risk Management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 303 Trading Bot Vectorized Backtesting Event-Based Backtesting Assessing Risk Backtesting Risk Measures Stop Loss Trailing Stop Loss Take Profit Conclusions References Python Code Finance Environment Trading Bot Backtesting Base Class Backtesting Class

304 308 311 318 322 324 326 328 332 332 333 333 335 339 342

12. Execution and Deployment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 Oanda Account Data Retrieval Order Execution Trading Bot Deployment Conclusions References Python Code Oanda Environment Vectorized Backtesting Oanda Trading Bot

Part V.

346 347 351 357 364 368 369 369 369 372 373

Outlook

13. AI-Based Competition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379 AI and Finance Lack of Standardization Education and Training Fight for Resources Market Impact Competitive Scenarios Risks, Regulation, and Oversight

380 382 383 385 386 387 388

Table of Contents

|

vii

Conclusions References

391 392

14. Financial Singularity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395 Notions and Definitions What Is at Stake? Paths to Financial Singularity Orthogonal Skills and Resources Scenarios Before and After Star Trek or Star Wars Conclusions References

396 396 400 401 402 403 404 404

Part VI. Appendixes A. Interactive Neural Networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407 B. Neural Network Classes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425 C. Convolutional Neural Networks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 447

viii

|

Table of Contents

Preface

Will alpha eventually go to zero for every imaginable investment strategy? More funda‐ mentally, is the day approaching when, thanks to so many smart people and smarter computers, financial markets really do become perfect, and we can just sit back, relax, and assume that all assets are priced correctly? —Robert Shiller (2015)

Artificial intelligence (AI) rose to become a key technology in the 2010s and is assumed to be the dominating technology in the 2020s. Spurred by technological innovations, algorithmic breakthroughs, availability of big data, and ever-increasing compute power, many industries are undergoing fundamental changes driven by AI. While media and public attention mostly focus on breakthroughs in areas such as gaming and self-driving cars, AI has also become a major technological force in the financial industry. However, it is safe to say that AI in finance is still at a nascent stage —as compared, for example, to industries such as web search or social media. This book sets out to cover a number of important aspects related to AI in finance. AI in finance is already a vast topic, and a single book needs to focus on selected aspects. Therefore, this book covers the basics first (see Part I and Part II). It then zooms in on discovering statistical inefficiencies in financial markets by the use of AI and, more specifically, neural networks (see Part III). Such inefficiencies—embodied by AI algo‐ rithms that successfully predict future market movements—are a prerequisite for the exploitation of economic inefficiencies through algorithmic trading (see Part IV). Being able to systematically exploit statistical and economic inefficiencies would prove contradictory to one of the established theories and cornerstones in finance: the efficient market hypothesis (EMH). The design of a successful trading bot can be considered the holy grail in finance to which AI might lead the way. This book con‐ cludes by discussing consequences of AI for the financial industry and the possibility of a financial singularity (see Part V). There is also a technical appendix that shows how to build neural networks from scratch based on plain Python code and provides additional examples for their application (see Part VI). ix

The problem of applying AI to finance is not too dissimilar to the problem of apply‐ ing AI to other fields. Some major breakthroughs in AI in the 2010s were made possi‐ ble by the application of reinforcement learning (RL) to playing arcade games, such as those from Atari published in the 1980s (see Mnih et al. 2013), and to board games, such as chess or Go (see Silver et al. 2016). Lessons learned from applying RL in gaming contexts, among other areas, are today applied to such challenging prob‐ lems as designing and building autonomous vehicles or improving medical diagnos‐ tics. Table P-1 compares the application of AI and RL in different domains. Table P-1. Comparison of AI in different domains Domain Agent Arcade games AI agent (software)

Goal Maximizing game score

Approach RL in virtual gaming environment

Reward Points and scores

Obstacle Risks Planning and None delayed rewards

Autonomous driving

Self-driving car (software + car)

Safely driving from location A to B

RL in virtual (gaming) environment, realworld test drives

Punishment for mistakes

Transition from virtual to physical world

Financial trading

Trading bot (software)

Maximizing long-term performance

RL in virtual trading Financial environment returns

Damaging property, harming people

Efficient markets Financial and competition losses

The beauty of training AI agents to play arcade games lies in the availability of a per‐ fect virtual learning environment1 and the absence of any kind of risk. With autono‐ mous vehicles, the major problem arises when transitioning from virtual learning environments—for example, a computer game such as Grand Theft Auto—to the physical world with a self-driving car navigating real streets populated by other cars and people. This leads to serious risks such as a car causing accidents or harming people. For a trading bot, RL can also be completely virtual, that is, in a simulated financial market environment. The major risks that arise from malfunctioning trading bots are financial losses and, on an aggregated level, potential systematic risks due to herding by trading bots. Overall, however, the financial domain seems like an ideal place to train, test, and deploy AI algorithms. Given the rapid developments in the field, it should even be possible for an interested and ambitious student, equipped with a notebook and internet access, to successfully apply AI in a financial trading context. Beyond the hardware and software improve‐ ments over recent years, this is due primarily to the rise of online brokers that supply

1 See the Arcade Learning Environment.

x

|

Preface

historical and real-time financial data and that allow the execution of financial trades via programmatic APIs. The book is structured in the following six parts. Part I The first part discusses central notions and algorithms of AI in general, such as supervised learning and neural networks (see Chapter 1). It also discusses the concept of superintelligence, which relates to an AI agent that possesses humanlevel intelligence and, in some domains, superhuman-level intelligence (see Chapter 2). Not every researcher in AI believes that superintelligence is possible in the foreseeable future. However, the discussion of this idea provides a valuable framework for discussing AI in general and AI for finance in particular. Part II The second part consists of four chapters and is about traditional, normative finance theory (see Chapter 3) and how the field is transformed by data-driven finance (see Chapter 4) and machine learning (ML) (see Chapter 5). Taken together, data-driven finance and ML give rise to a model-free, AI-first approach to finance, as discussed in Chapter 6. Part III The third part is about discovering statistical inefficiencies in financial markets by applying deep learning, neural networks, and reinforcement learning. The part covers dense neural networks (DNNs, see Chapter 7), recurrent neural net‐ works (RNNs, see Chapter 8), and algorithms from reinforcement learning (RL, see Chapter 9) that in turn often rely on DNNs to represent and approximate the optimal policy of the AI agent. Part IV The fourth part discusses how to exploit statistical inefficiencies through algo‐ rithmic trading. Topics are vectorized backtesting (see Chapter 10), event-based backtesting and risk management (see Chapter 11), and execution and deploy‐ ment of AI-powered algorithmic trading strategies (see Chapter 12). Part V The fifth part is about the consequences that arise from AI-based competition in the financial industry (see Chapter 13). It also discusses the possibility of a finan‐ cial singularity, a point in time at which AI agents would dominate all aspects of finance as we know it. The discussion in this context focuses on artificial finan‐ cial intelligences as trading bots that consistently generate trading profits above any human or institutional benchmark (see Chapter 14).

Preface

|

xi

Part VI The Appendix contains Python code for interactive neural network training (see Appendix A), classes for simple and shallow neural networks that are imple‐ mented from scratch based on plain Python code (see Appendix B), and an example of how to use convolutional neural networks (CNNs) for financial time series prediction (see Appendix C).

Author’s Note The application of AI to financial trading is still a nascent field, although at the time of writing there are a number of other books available that cover this topic to some extent. Many of these publications, however, fail to show what it means to economi‐ cally exploit statistical inefficiencies. Some hedge funds already claim to exclusively rely on machine learning to manage their investors’ capital. A prominent example is The Voleon Group, a hedge fund that reported more than $6 billion in assets under management at the end of 2019 (see Lee and Karsh 2020). The difficulty of relying on machine learning to outsmart the finan‐ cial markets is reflected in the fund’s performance of 7% for 2019, a year during which the S&P 500 stock index rose by almost 30%. This book is based on years of practical experience in developing, backtesting, and deploying AI-powered algorithmic trading strategies. The approaches and examples presented are mostly based on my own research since the field is, by nature, not only nascent, but also rather secretive. The exposition and the style throughout this book are relentlessly practical, and in many instances the concrete examples are lacking proper theoretical support and/or comprehensive empirical evidence. This book even presents some applications and examples that might be vehemently criticized by experts in finance and/or machine learning. For example, some experts in machine and deep learning, such as François Chollet (2017), outright doubt that prediction in financial markets is possible. Certain experts in finance, such as Robert Shiller (2015), doubt that there will ever be something like a financial singularity. Others active at the intersection of the two domains, such as Marcos López de Prado (2018), argue that the use of machine learning for financial trading and investing requires an industrial-scale effort with large teams and huge budgets. This book does not try to provide a balanced view of or a comprehensive set of refer‐ ences for all the topics covered. The presentation is driven by the personal opinions and experiences of the author, as well as by practical considerations when providing concrete examples and Python code. Many of the examples are also chosen and tweaked to drive home certain points or to show encouraging results. Therefore, it can certainly be argued that results from many examples presented in the book suffer from data snooping and overfitting (for a discussion of these topics, see Hilpisch 2020, ch. 4). xii

| Preface

The major goal of this book is to empower the reader to use the code examples in the book as a framework to explore the exciting space of AI applied to financial trading. To achieve this goal, the book relies throughout on a number of simplifying assump‐ tions and primarily on financial time series data and features derived directly from such data. In practical applications, a restriction to financial time series data is of course not necessary—a great variety of other types of data and data sources could be used as well. This book’s approach to deriving features implicitly assumes that financial time series and features derived from them show patterns that, at least to some extent, per‐ sist over time and that can be used to predict the direction of future price movements. Against this background, all examples and code presented in this book are technical and illustrative in nature and do not represent any recommendation or investment advice. For those who want to deploy approaches and algorithmic trading strategies presen‐ ted in this book, my book Python for Algorithmic Trading: From Idea to Cloud Deploy‐ ment (O’Reilly) provides more process-oriented and technical details. The two books complement each other in many respects. For readers who are just getting started with Python for finance or who are seeking a refresher and reference manual, my book Python for Finance: Mastering Data-Driven Finance (O’Reilly) covers a compre‐ hensive set of important topics and fundamental skills in Python as applied to the financial domain.

References Papers and books cited in the preface: Chollet, François. 2017. Deep Learning with Python. Shelter Island: Manning. Hilpisch, Yves. 2018. Python for Finance: Mastering Data-Driven Finance. 2nd ed. Sebastopol: O’Reilly. ⸻. 2020. Python for Algorithmic Trading: From Idea to Cloud Deployment. Sebas‐ topol: O’Reilly. Lee, Justina and Melissa Karsh. 2020. “Machine-Learning Hedge Fund Voleon Group Returns 7% in 2019.” Bloomberg, January 21, 2020. https://oreil.ly/TOQiv. López de Prado, Marcos. 2018. Advances in Financial Machine Learning. Hoboken, NJ: John Wiley & Sons. Mnih, Volodymyr et al. 2013. “Playing Atari with Deep Reinforcement Learning.” arXiv. December 19. https://oreil.ly/-pW-1.

Preface

|

xiii

Shiller, Robert. 2015. “The Mirage of the Financial Singularity.” Yale Insights. July 16. https://oreil.ly/VRkP3. Silver, David et al. 2016. “Mastering the Game of Go with Deep Neural Networks and Tree Search.” Nature 529 (January): 484-489.

Conventions Used in This Book The following typographical conventions are used in this book: Italic Indicates new terms, URLs, email addresses, filenames, and file extensions. Constant width

Used for program listings, as well as within paragraphs to refer to program ele‐ ments such as variable or function names, databases, data types, environment variables, statements, and keywords. Constant width bold

Shows commands or other text that should be typed literally by the user. Constant width italic

Shows text that should be replaced with user-supplied values or by values deter‐ mined by context. This element signifies a tip or suggestion.

This element signifies a general note.

This element indicates important information.

This element indicates a warning or caution.

xiv

|

Preface

Using Code Examples You can access and execute the code that accompanies the book on the Quant Plat‐ form at https://aiif.pqp.io, for which only a free registration is required. If you have a technical question or a problem using the code examples, please send an email to [email protected]. This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require per‐ mission. We appreciate, but generally do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example, this book may be attrib‐ uted as: “Artificial Intelligence in Finance by Yves Hilpisch (O’Reilly). Copyright 2021 Yves Hilpisch, 978-1-492-05543-3.” If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at [email protected].

O’Reilly Online Learning For more than 40 years, O’Reilly Media has provided technol‐ ogy and business training, knowledge, and insight to help companies succeed. Our unique network of experts and innovators share their knowledge and expertise through books, articles, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, visit http://oreilly.com.

Preface

|

xv

How to Contact Us Please address comments and questions concerning this book to the publisher: O’Reilly Media, Inc. 1005 Gravenstein Highway North Sebastopol, CA 95472 800-998-9938 (in the United States or Canada) 707-829-0515 (international or local) 707-829-0104 (fax) We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/ai-in-finance. Email [email protected] to comment or ask technical questions about this book. For news and information about our books and courses, visit http://oreilly.com. Find us on Facebook: http://facebook.com/oreilly Follow us on Twitter: http://twitter.com/oreillymedia Watch us on YouTube: http://www.youtube.com/oreillymedia

Acknowledgments I want to thank the technical reviewers—Margaret Maynard-Reid, Dr. Tim Nugent, and Dr. Abdullah Karasan—who did a great job in helping me improve the contents of the book. Delegates of the Certificate Programs in Python for Computational Finance and Algorithmic Trading also helped improve this book. Their ongoing feedback has enabled me to weed out errors and mistakes and refine the code and notebooks used in our online training classes and now, finally, in this book. The same holds true for the team members of The Python Quants and The AI Machine. In particular, Michael Schwed, Ramanathan Ramakrishnamoorthy, and Prem Jebaseelan support me in numerous ways. They are the ones who assist me with the difficult technical problems that arise during the writing of a book like this one. I would also like to thank the whole team at O’Reilly Media—especially Michelle Smith, Corbin Collins, Victoria DeRose, and Danny Elfanbaum—for making it all happen and helping me refine the book in so many ways. Of course, all remaining errors are mine alone.

xvi

|

Preface

Furthermore, I would also like to thank the team at Refinitiv—in particular, Jason Ramchandani—for providing ongoing support and access to financial data. The major data files used throughout the book and made available to the readers were received in one way or another from Refinitiv’s data APIs. Of course, everybody making use of artificial intelligence and machine learning today benefits from the achievements and contributions of so many others. Therefore, we should always recall what Sir Isaac Newton wrote in 1675: “If I have seen further it is by standing on the shoulders of Giants.” In that sense, a big thank you to all the researchers and open source maintainers contributing to the field. Finally, special thanks go to my family, who support me all year round in my business and book-writing activities. In particular, I thank my wife Sandra for relentlessly tak‐ ing care of us all and for providing us with a home and environment that we all love so much. I dedicate this book to my lovely wife Sandra and my wonderful son Henry.

Preface

|

xvii

PART I

Machine Intelligence

Today’s algorithmic trading programs are relatively simple and make only limited use of AI. This is sure to change. —Murray Shanahan (2015)

This part is about artificial intelligence (AI) in general: artificial in the sense that the intelligence is not displayed by a biological organism but rather by a machine, and intelligence as defined by AI researcher Max Tegmark as the “ability to accomplish complex goals.” This part introduces central notions and algorithms from the AI field, gives examples of major recent breakthroughs, and discusses aspects of superintelli‐ gence. It consists of two chapters: • Chapter 1 introduces general notions, ideas, and definitions from the field of AI. It also provides several Python examples of how different algorithms can be applied in practice. • Chapter 2 discusses concepts and topics related to artificial general intelligence (AGI) and superintelligence (SI). These types of intelligence relate to AI agents that have reached at least human-level intelligence in all domains and superhuman intelligence in certain domains.

CHAPTER 1

Artificial Intelligence

This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away. —David Silver et al. (2016)

This chapter introduces general notions, ideas, and definitions from the field of artifi‐ cial intelligence (AI) for the purposes of this book. It also provides worked-out exam‐ ples for different types of major learning algorithms. In particular, “Algorithms” on page 3 takes a broad perspective and categorizes types of data, types of learning, and types of problems typically encountered in an AI context. This chapter also presents examples for unsupervised and reinforcement learning. “Neural Networks” on page 9 jumps right into the world of neural networks, which not only are central to what fol‐ lows in later chapters of the book but also have proven to be among the most power‐ ful algorithms AI has to offer nowadays. “Importance of Data” on page 22 discusses the importance of data volume and variety in the context of AI.

Algorithms This section introduces basic notions from the field of AI relevant to this book. It dis‐ cusses the different types of data, learning, problems, and approaches that can be sub‐ sumed under the general term AI. Alpaydin (2016) provides an informal introduction to and overview of many of the topics covered only briefly in this section, along with many examples.

3

Types of Data Data in general has two major components: Features Features data (or input data) is data that is given as input to an algorithm. In a financial context, this might be, for example, the income and the savings of a potential debtor. Labels Labels data (or output data) is data that is given as the relevant output to be learned, for example, by a supervised learning algorithm. In a financial context, this might be the creditworthiness of a potential debtor.

Types of Learning There are three major types of learning algorithms: Supervised learning (SL) These are algorithms that learn from a given sample data set of features (input) and labels (output) values. The next section presents examples for such algo‐ rithms, like ordinary least-squares (OLS) regression and neural networks. The purpose of supervised learning is to learn the relationship between the input and output values. In finance, such algorithms might be trained to predict whether a potential debtor is creditworthy or not. For the purposes of this book, these are the most important types of algorithms. Unsupervised learning (UL) These are algorithms that learn from a given sample data set of features (input) values only, often with the goal of finding structure in the data. They are sup‐ posed to learn about the input data set, given, for example, some guiding param‐ eters. Clustering algorithms fall into that category. In a financial context, such algorithms might cluster stocks into certain groups. Reinforcement learning (RL) These are algorithms that learn from trial and error by receiving a reward for tak‐ ing an action. They update an optimal action policy according to what rewards and punishments they receive. Such algorithms are, for example, used for envi‐ ronments where actions need to be taken continuously and rewards are received immediately, such as in a computer game. Because supervised learning is addressed in the subsequent section in some detail, brief examples will illustrate unsupervised learning and reinforcement learning.

4

|

Chapter 1: Artificial Intelligence

Unsupervised Learning Simply speaking, a k-means clustering algorithm sorts n observations into k clusters. Each observation belongs to the cluster to which its mean (center) is nearest. The fol‐ lowing Python code generates sample data for which the features data is clustered. Figure 1-1 visualizes the clustered sample data and also shows that the scikit-learn KMeans algorithm used here has identified the clusters perfectly. The coloring of the dots is based on what the algorithm has learned.1 In [1]: import numpy as np import pandas as pd from pylab import plt, mpl plt.style.use('seaborn') mpl.rcParams['savefig.dpi'] = 300 mpl.rcParams['font.family'] = 'serif' np.set_printoptions(precision=4, suppress=True) In [2]: from sklearn.cluster import KMeans from sklearn.datasets import make_blobs In [3]: x, y = make_blobs(n_samples=100, centers=4, random_state=500, cluster_std=1.25) In [4]: model = KMeans(n_clusters=4, random_state=0) In [5]: model.fit(x) Out[5]: KMeans(n_clusters=4, random_state=0) In [6]: y_ = model.predict(x) In [7]: y_ Out[7]: array([3, 1, 3, 1, 0,

3, 2, 1, 1, 0,

1, 1, 2, 1, 0,

2, 1, 0, 2, 3,

1, 0, 2, 3, 3,

1, 0, 0, 1, 3,

3, 1, 3, 2, 3,

2, 3, 0, 0, 0,

1, 2, 1, 2, 3,

2, 1, 0, 3, 1,

In [8]: plt.figure(figsize=(10, 6)) plt.scatter(x[:, 0], x[:, 1], c=y_,

2, 1, 1, 2, 0,

3, 2, 0, 0, 3, 2, 0, 1, 3, 1, 3, 2, 3, 1, 2, 0, 3, 1, 0, 2, 2, 1, 3, 1, 0], dtype=int32)

0, 2, 0, 3,

2, 2, 3, 2,

0, 1, 2, 2,

0, 0, 3, 3,

3, 0, 0, 2,

cmap='coolwarm');

A sample data set is created with clustered features data. A KMeans model object is instantiated, fixing the number of clusters. The model is fitted to the features data.

1 For details, see sklearn.cluster.KMeans and VanderPlas (2017, ch. 5).

Algorithms

|

5

The predictions are generated given the fitted model. The predictions are numbers from 0 to 3, each representing one cluster.

Figure 1-1. Unsupervised learning of clusters Once an algorithm such as KMeans is trained, it can, for instance, predict the cluster for a new (not yet seen) combination of features values. Assume that such an algo‐ rithm is trained on features data that describes potential and real debtors of a bank. It might learn about the creditworthiness of potential debtors by generating two clus‐ ters. New potential debtors can then be sorted into a certain cluster: “creditworthy” versus “not creditworthy.”

Reinforcement learning The following example is based on a coin tossing game that is played with a coin that lands 80% of the time on heads and 20% of the time on tails. The coin tossing game is heavily biased to emphasize the benefits of learning as compared to an uninformed baseline algorithm. The baseline algorithm, which bets randomly and equally distrib‐ utes on heads and tails, achieves a total reward of around 50, on average, per epoch of 100 bets played: In [9]: ssp = [1, 1, 1, 1, 0] In [10]: asp = [1, 0] In [11]: def epoch(): tr = 0 for _ in range(100): a = np.random.choice(asp) s = np.random.choice(ssp)

6

|

Chapter 1: Artificial Intelligence

if a == s: tr += 1 return tr In [12]: rl = np.array([epoch() for _ in range(15)]) rl Out[12]: array([53, 55, 50, 48, 46, 41, 51, 49, 50, 52, 46, 47, 43, 51, 52]) In [13]: rl.mean() Out[13]: 48.93333333333333

The state space (1 = heads, 0 = tails). The action space (1 = bet on heads, 0 = bet on tails). An action is randomly chosen from the action space. A state is randomly chosen from the state space. The total reward tr is increased by one if the bet is correct. The game is played for a number of epochs; each epoch is 100 bets. The average total reward of the epochs played is calculated. Reinforcement learning tries to learn from what is observed after an action is taken, usually based on a reward. To keep things simple, the following learning algorithm only keeps track of the states that are observed in each round insofar as they are appended to the action space list object. In this way, the algorithm learns the bias in the game, though maybe not perfectly. By randomly sampling from the updated action space, the bias is reflected because naturally the bet will more often be heads. Over time, heads is chosen, on average, around 80% of the time. The average total reward of around 65 reflects the improvement of the learning algorithm as compared to the uninformed baseline algorithm: In [14]: ssp = [1, 1, 1, 1, 0] In [15]: def epoch(): tr = 0 asp = [0, 1] for _ in range(100): a = np.random.choice(asp) s = np.random.choice(ssp) if a == s: tr += 1 asp.append(s) return tr

Algorithms

|

7

In [16]: rl = np.array([epoch() for _ in range(15)]) rl Out[16]: array([64, 65, 77, 65, 54, 64, 71, 64, 57, 62, 69, 63, 61, 66, 75]) In [17]: rl.mean() Out[17]: 65.13333333333334

Resets the action space before starting (over) Adds the observed state to the action space

Types of Tasks Depending on the type of labels data and the problem at hand, two types of tasks to be learned are important: Estimation Estimation (or approximation, regression) refers to the cases in which the labels data is real-valued (continuous); that is, it is technically represented as floating point numbers. Classification Classification refers to the cases in which the labels data consists of a finite num‐ ber of classes or categories that are typically represented by discrete values (posi‐ tive natural numbers), which in turn are represented technically as integers. The following section provides examples for both types of tasks.

Types of Approaches Some more definitions might be in order before finishing this section. This book fol‐ lows the common differentiation between the following three major terms: Artificial intelligence (AI) AI encompasses all types of learning (algorithms), as defined before, and some more (for example, expert systems). Machine learning (ML) ML is the discipline of learning relationships and other information about given data sets based on an algorithm and a measure of success; a measure of success might, for example, be the mean-squared error (MSE) given labels values and output values to be estimated and the predicted values from the algorithm. ML is a sub-set of AI.

8

|

Chapter 1: Artificial Intelligence

Deep learning (DL) DL encompasses all algorithms based on neural networks. The term deep is usu‐ ally only used when the neural network has more than one hidden layer. DL is a sub-set of machine learning and so is therefore also a sub-set of AI. DL has proven useful for a number of broad problem areas. It is suited for estimation and classification tasks, as well as for RL. In many cases, DL-based approaches perform better than alternative algorithms, such as logistic regression or kernel-based ones, like support vector machines.2 That is why this book mainly focuses on DL. DL approaches used include dense neural networks (DNNs), recurrent neural networks (RNNs), and convolutional neural networks (CNNs). More details appear in later chapters, particularly in Part III.

Neural Networks The previous sections provide a broader overview of algorithms in AI. This section shows how neural networks fit in. A simple example will illustrate what characterizes neural networks in comparison to traditional statistical methods, such as ordinary least-squares (OLS) regression. The example starts with mathematics and then uses linear regression for estimation (or function approximation) and finally applies neu‐ ral networks to accomplish the estimation. The approach taken here is a supervised learning approach where the task is to estimate labels data based on features data. This section also illustrates the use of neural networks in the context of classification problems.

OLS Regression Assume that a mathematical function is given as follows: f :ℝ

1 ℝ, y = 2x2 − x3 3

Such a function transforms an input value x to an output value y. Or it transforms a series of input values x1, x2, ..., xN into a series of output values y1, y2, ..., yN . The fol‐ lowing Python code implements the mathematical function as a Python function and creates a number of input and output values. Figure 1-2 plots the output values against the input values: In [18]: def f(x): return 2 * x ** 2 - x ** 3 / 3

2 For details, see VanderPlas (2017, ch. 5).

Neural Networks

|

9

In [19]: x = np.linspace(-2, 4, 25) x Out[19]: array([-2. , -1.75, -1.5 , -1.25, -1. , -0.75, -0.5 , -0.25, 0.25, 0.5 , 0.75, 1. , 1.25, 1.5 , 1.75, 2. , 2.5 , 2.75, 3. , 3.25, 3.5 , 3.75, 4. ]) In [20]: y = f(x) y Out[20]: array([10.6667, 7.9115, 5.625 , 3.776 , 2.3333, 1.2656, 0.1302, 0. , 0.1198, 0.4583, 0.9844, 1.6667, 3.375 , 4.3385, 5.3333, 6.3281, 7.2917, 8.1927, 9.6823, 10.2083, 10.5469, 10.6667])

0. , 2.25,

0.5417, 2.474 , 9. ,

In [21]: plt.figure(figsize=(10, 6)) plt.plot(x, y, 'ro');

The mathematical function as a Python function The input values The output values

Figure 1-2. Output values against input values Whereas in the mathematical example the function comes first, the input data sec‐ ond, and the output data third, the sequence is different in statistical learning. Assume that the previous input values and output values are given. They represent the sample (data). The problem in statistical regression is to find a function that approximates the functional relationship between the input values (also called the independent values) and the output values (also called the dependent values) as well as possible.

10

|

Chapter 1: Artificial Intelligence

Assume simple OLS linear regression. In this case, the functional relationship between the input and output values is assumed to be linear, and the problem is to find optimal parameters α and β for the following linear equation: f :ℝ

ℝ, y = α + βx

For given input values x1, x2, ..., xN and output values y1, y2, ..., yN , optimal in this case means that they minimize the mean squared error (MSE) between the real output values and the approximated output values: min α, β

1N yn − f xn N∑ n

2

For the case of simple linear regression, the solution α*, β* is known in closed form, as shown in the following equation. Bars on the variables indicate sample mean val‐ ues: Cov x, y Var x α* = y − βx β* =

The following Python code calculates the optimal parameter values, linearly estimates (approximates) the output values, and plots the linear regression line alongside the sample data (see Figure 1-3). The linear regression approach does not work too well here in approximating the functional relationship. This is confirmed by the relatively high MSE value: In [22]: beta = np.cov(x, y, ddof=0)[0, 1] / np.var(x) beta Out[22]: 1.0541666666666667 In [23]: alpha = y.mean() - beta * x.mean() alpha Out[23]: 3.8625000000000003 In [24]: y_ = alpha + beta * x In [25]: MSE = ((y - y_) ** 2).mean() MSE Out[25]: 10.721953125 In [26]: plt.figure(figsize=(10, 6)) plt.plot(x, y, 'ro', label='sample data') plt.plot(x, y_, lw=3.0, label='linear regression') plt.legend();

Neural Networks

|

11

Calculation of optimal β Calculation of optimal α Calculation of estimated output values Calculation of the MSE given the approximation

Figure 1-3. Sample data and linear regression line How can the MSE value be improved (decreased)—maybe even to 0, that is, to a “per‐ fect estimation?” Of course, OLS regression is not constrained to a simple linear rela‐ tionship. In addition to the constant and linear terms, higher order monomials, for instance, can be easily added as basis functions. To this end, compare the regression results shown in Figure 1-4 and the following code that creates the figure. The improvements that come from using quadratic and cubic monomials as basis func‐ tions are obvious and also are numerically confirmed by the calculated MSE values. For basis functions up to and including the cubic monomial, the estimation is perfect, and the functional relationship is perfectly recovered: In [27]: plt.figure(figsize=(10, 6)) plt.plot(x, y, 'ro', label='sample data') for deg in [1, 2, 3]: reg = np.polyfit(x, y, deg=deg) y_ = np.polyval(reg, x) MSE = ((y - y_) ** 2).mean() print(f'deg={deg} | MSE={MSE:.5f}') plt.plot(x, np.polyval(reg, x), label=f'deg={deg}') plt.legend(); deg=1 | MSE=10.72195 deg=2 | MSE=2.31258

12

| Chapter 1: Artificial Intelligence

deg=3 | MSE=0.00000 In [28]: reg Out[28]: array([-0.3333,

2.

,

0.

, -0.

])

Regression step Approximation step MSE calculation Optimal (“perfect”) parameter values

Figure 1-4. Sample data and OLS regression lines Exploiting the knowledge of the form of the mathematical function to be approxima‐ ted and accordingly adding more basis functions to the regression leads to a “perfect approximation.” That is, the OLS regression recovers the exact factors of the quad‐ ratic and cubic part, respectively, of the original function.

Estimation with Neural Networks However, not all relationships are of this kind. This is where, for instance, neural net‐ works can help. Without going into the details, neural networks can approximate a wide range of functional relationships. Knowledge of the form of the relationship is generally not required.

Neural Networks

|

13

Scikit-learn The following Python code uses the MLPRegressor class of scikit-learn, which implements a DNN for estimation. DNNs are sometimes also called multi-layer per‐ ceptron (MLP).3 The results are not perfect, as Figure 1-5 and the MSE illustrate. However, they are quite good already for the simple configuration used: In [29]: from sklearn.neural_network import MLPRegressor In [30]: model = MLPRegressor(hidden_layer_sizes=3 * [256], learning_rate_init=0.03, max_iter=5000) In [31]: model.fit(x.reshape(-1, 1), y) Out[31]: MLPRegressor(hidden_layer_sizes=[256, 256, 256], learning_rate_init=0.03, max_iter=5000) In [32]: y_ = model.predict(x.reshape(-1, 1)) In [33]: MSE = ((y - y_) ** 2).mean() MSE Out[33]: 0.021662355744355866 In [34]: plt.figure(figsize=(10, 6)) plt.plot(x, y, 'ro', label='sample data') plt.plot(x, y_, lw=3.0, label='dnn estimation') plt.legend();

Instantiates the MLPRegressor object Implements the fitting or learning step Implements the prediction step Just having a look at the results in Figure 1-4 and Figure 1-5, one might assume that the methods and approaches are not too dissimilar after all. However, there is a fun‐ damental difference worth highlighting. Although the OLS regression approach, as shown explicitly for the simple linear regression, is based on the calculation of certain well-specified quantities and parameters, the neural network approach relies on incre‐ mental learning. This in turn means that a set of parameters, the weights within the neural network, are first initialized randomly and then adjusted gradually given the differences between the neural network output and the sample output values. This approach lets you retrain (update) a neural network incrementally.

3 For details, see sklearn.neural_network.MLPRegressor. For more background, see Goodfellow et al. (2016,

ch. 6).

14

| Chapter 1: Artificial Intelligence

Figure 1-5. Sample data and neural network–based estimations

Keras The next example uses a sequential model with the Keras deep learning package.4 The model is fitted, or trained, for 100 epochs. The procedure is repeated for five rounds. After every such round, the approximation by the neural network is updated and plotted. Figure 1-6 shows how the approximation gradually improves with every round. This is also reflected in the decreasing MSE values. The end result is not per‐ fect, but again, it is quite good given the simplicity of the model: In [35]: import tensorflow as tf tf.random.set_seed(100) In [36]: from keras.layers import Dense from keras.models import Sequential Using TensorFlow backend. In [37]: model = Sequential() model.add(Dense(256, activation='relu', input_dim=1)) model.add(Dense(1, activation='linear')) model.compile(loss='mse', optimizer='rmsprop') In [38]: ((y - y_) ** 2).mean() Out[38]: 0.021662355744355866 In [39]: plt.figure(figsize=(10, 6)) plt.plot(x, y, 'ro', label='sample data') for _ in range(1, 6):

4 For details, see Chollet (2017, ch. 3).

Neural Networks

|

15

model.fit(x, y, epochs=100, verbose=False) y_ = model.predict(x) MSE = ((y - y_.flatten()) ** 2).mean() print(f'round={_} | MSE={MSE:.5f}') plt.plot(x, y_, '--', label=f'round={_}') plt.legend(); round=1 | MSE=3.09714 round=2 | MSE=0.75603 round=3 | MSE=0.22814 round=4 | MSE=0.11861 round=5 | MSE=0.09029

Instantiates the Sequential model object Adds a densely connected hidden layer with rectified linear unit (ReLU) activation5 Adds the output layer with linear activation Compiles the model for usage Trains the neural network for a fixed number of epochs Implements the approximation step Calculates the current MSE Plots the current approximation results Roughly speaking, one can say that the neural network does almost as well in the esti‐ mation as the OLS regression, which delivers a perfect result. Therefore, why use neu‐ ral networks at all? A more comprehensive answer might need to come later in this book, but a somewhat different example might give some hint. Consider instead the previous sample data set, as generated from a well-defined mathematical function, now a random sample data set, for which both features and labels are randomly chosen. Of course, this example is for illustration and does not allow for a deep interpretation.

5 For details on activation functions with Keras, see https://keras.io/activations.

16

| Chapter 1: Artificial Intelligence

Figure 1-6. Sample data and estimations after multiple training rounds The following code generates the random sample data set and creates the OLS regres‐ sion estimation based on a varying number of monomial basis functions. Figure 1-7 visualizes the results. Even for the highest number of monomials in the example, the estimation results are still not too good. The MSE value is accordingly relatively high: In [40]: np.random.seed(0) x = np.linspace(-1, 1) y = np.random.random(len(x)) * 2 - 1 In [41]: plt.figure(figsize=(10, 6)) plt.plot(x, y, 'ro', label='sample data') for deg in [1, 5, 9, 11, 13, 15]: reg = np.polyfit(x, y, deg=deg) y_ = np.polyval(reg, x) MSE = ((y - y_) ** 2).mean() print(f'deg={deg:2d} | MSE={MSE:.5f}') plt.plot(x, np.polyval(reg, x), label=f'deg={deg}') plt.legend(); deg= 1 | MSE=0.28153 deg= 5 | MSE=0.27331 deg= 9 | MSE=0.25442 deg=11 | MSE=0.23458 deg=13 | MSE=0.22989 deg=15 | MSE=0.21672

The results for the OLS regression are not too surprising. OLS regression in this case assumes that the approximation can be achieved through an appropriate combination of a finite number of basis functions. Since the sample data set has been generated randomly, the OLS regression does not perform well in this case.

Neural Networks

|

17

Figure 1-7. Random sample data and OLS regression lines What about neural networks? The application is as straightforward as before and yields estimations as shown in Figure 1-8. While the end result is not perfect, it is obvious that the neural network performs better than the OLS regression in estimat‐ ing the random label values from the random features values. Given its architecture, however, the neural network has almost 200,000 trainable parameters (weights), which offers relatively high flexibility, particularly when compared to the OLS regres‐ sion, for which a maximum of 15 + 1 parameters are used: In [42]: model = Sequential() model.add(Dense(256, activation='relu', input_dim=1)) for _ in range(3): model.add(Dense(256, activation='relu')) model.add(Dense(1, activation='linear')) model.compile(loss='mse', optimizer='rmsprop') In [43]: model.summary() Model: "sequential_2" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense_3 (Dense) (None, 256) 512 _________________________________________________________________ dense_4 (Dense) (None, 256) 65792 _________________________________________________________________ dense_5 (Dense) (None, 256) 65792 _________________________________________________________________ dense_6 (Dense) (None, 256) 65792 _________________________________________________________________ dense_7 (Dense) (None, 1) 257 =================================================================

18

| Chapter 1: Artificial Intelligence

Total params: 198,145 Trainable params: 198,145 Non-trainable params: 0 _________________________________________________________________ In [44]: %%time plt.figure(figsize=(10, 6)) plt.plot(x, y, 'ro', label='sample data') for _ in range(1, 8): model.fit(x, y, epochs=500, verbose=False) y_ = model.predict(x) MSE = ((y - y_.flatten()) ** 2).mean() print(f'round={_} | MSE={MSE:.5f}') plt.plot(x, y_, '--', label=f'round={_}') plt.legend(); round=1 | MSE=0.13560 round=2 | MSE=0.08337 round=3 | MSE=0.06281 round=4 | MSE=0.04419 round=5 | MSE=0.03329 round=6 | MSE=0.07676 round=7 | MSE=0.00431 CPU times: user 30.4 s, sys: 4.7 s, total: 35.1 s Wall time: 13.6 s

Multiple hidden layers are added. Network architecture and number of trainable parameters are shown.

Figure 1-8. Random sample data and neural network estimations

Neural Networks

|

19

Classification with Neural Networks Another benefit of neural networks is that they can be easily used for classification tasks as well. Consider the following Python code that implements a classification using a neural network based on Keras. The binary features data and labels data are generated randomly. The major adjustment to be made modeling-wise is to change the activation function from the output layer to sigmoid from linear. More details on this appear in later chapters. The classification is not perfect. However, it reaches a high level of accuracy. How the accuracy, expressed as the relationship between correct results to all label values, changes with the number of training epochs is shown in Figure 1-9. The accuracy starts out low and then improves step-wise, though not necessarily with every step: In [45]: f = 5 n = 10 In [46]: np.random.seed(100) In [47]: x = np.random.randint(0, 2, (n, f)) x Out[47]: array([[0, 0, 1, 1, 1], [1, 0, 0, 0, 0], [0, 1, 0, 0, 0], [0, 1, 0, 0, 1], [0, 1, 0, 0, 0], [1, 1, 1, 0, 0], [1, 0, 0, 1, 1], [1, 1, 1, 0, 0], [1, 1, 1, 1, 1], [1, 1, 1, 0, 1]]) In [48]: y = np.random.randint(0, 2, n) y Out[48]: array([1, 1, 0, 0, 1, 1, 0, 1, 0, 1]) In [49]: model = Sequential() model.add(Dense(256, activation='relu', input_dim=f)) model.add(Dense(1, activation='sigmoid')) model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['acc']) In [50]: model.fit(x, y, epochs=50, verbose=False) Out[50]: In [51]: y_ = np.where(model.predict(x).flatten() > 0.5, 1, 0) y_ Out[51]: array([1, 1, 0, 0, 0, 1, 0, 1, 0, 1], dtype=int32)

20

|

Chapter 1: Artificial Intelligence

In [52]: y == y_ Out[52]: array([ True, True, True])

True,

True, False,

True,

True,

True,

True,

In [53]: res = pd.DataFrame(model.history.history) In [54]: res.plot(figsize=(10, 6));

Creates random features data Creates random labels data Defines the activation function for the output layer as sigmoid Defines the loss function to be binary_crossentropy6 Compares the predicted values with the labels data Plots the loss function and accuracy values for every training step

Figure 1-9. Classification accuracy and loss against number of epochs

6 The loss function calculates the prediction error of the neural network (or other ML algorithms). Binary cross

entropy is an appropriate loss function for binary classification problems, while the mean squared error (MSE) is, for example, appropriate for estimation problems. For details on loss functions with Keras, see https:// keras.io/losses.

Neural Networks

|

21

The examples in this section illustrate some fundamental characteristics of neural networks as compared to OLS regression: Problem-agnostic The neural network approach is agnostic when it comes to estimating and classi‐ fying label values, given a set of feature values. Statistical methods, such as OLS regression, might perform well for a smaller set of problems, but not too well or not at all for others. Incremental learning The optimal weights within a neural network, given a target measure of success, are learned incrementally based on a random initialization and incremental improvements. These incremental improvements are achieved by considering the differences between the predicted values and the sample label values and back‐ propagating weights updates through the neural network. Universal approximation There are strong mathematical theorems showing that neural networks (even with one hidden layer only) can approximate almost any function.7 These characteristics might justify why this book puts neural networks at the core with regard to the algorithms used. Chapter 2 discusses more good reasons.

Neural Networks Neural networks are good at learning relationships between input and output data. They can be applied to a number of problem types, such as estimation in the presence of complex relationships or classification, for which traditional statistical methods are not well suited.

Importance of Data The example at the end of the previous section shows that neural networks are capa‐ ble of solving classification problems quite well. The neural network with one hidden layer reaches a high degree of accuracy on the given data set, or in-sample. However, what about the predictive power of a neural network? This hinges significantly on the volume and variety of the data available to train the neural network. Another numeri‐ cal example, based on larger data sets, will illustrate this point.

7 See, for example, Kratsios (2019).

22

| Chapter 1: Artificial Intelligence

Small Data Set Consider a random sample data set similar to the one used before in the classification example, but with more features and more samples. Most algorithms used in AI are about pattern recognition. In the following Python code, the number of binary fea‐ tures defines the number of possible patterns about which the algorithm can learn something. Given that the labels data is also binary, the algorithm tries to learn whether a 0 or 1 is more likely given a certain pattern, say [0, 0, 1, 1, 1, 1, 0, 0, 0, 0]. Because all numbers are randomly chosen with equal probability, there is not that much to learn beyond the fact that the labels 0 and 1 are equally likely no matter what (random) pattern is observed. Therefore, a baseline prediction algorithm should be accurate about 50% of the time, no matter what (random) pattern it is presented with: In [55]: f = 10 n = 250 In [56]: np.random.seed(100) In [57]: x = np.random.randint(0, 2, (n, x[:4] Out[57]: array([[0, 0, 1, 1, 1, 1, 0, 0, [0, 1, 0, 0, 0, 0, 1, 0, [0, 1, 0, 0, 0, 1, 1, 1, [1, 0, 0, 1, 1, 1, 1, 1,

f)) 0, 0, 0, 0,

0], 1], 0], 0]])

In [58]: y = np.random.randint(0, 2, n) y[:4] Out[58]: array([0, 1, 0, 0]) In [59]: 2 ** f Out[59]: 1024

Features data Labels data Number of patterns In order to proceed, the raw data is put into a pandas DataFrame object, which simpli‐ fies certain operations and analyses: In [60]: fcols = [f'f{_}' for _ in range(f)] fcols Out[60]: ['f0', 'f1', 'f2', 'f3', 'f4', 'f5', 'f6', 'f7', 'f8', 'f9'] In [61]: data = pd.DataFrame(x, columns=fcols) data['l'] = y

Importance of Data

|

23

In [62]: data.info()

RangeIndex: 250 entries, 0 to 249 Data columns (total 11 columns): # Column Non-Null Count Dtype --- ------ -------------- ----0 f0 250 non-null int64 1 f1 250 non-null int64 2 f2 250 non-null int64 3 f3 250 non-null int64 4 f4 250 non-null int64 5 f5 250 non-null int64 6 f6 250 non-null int64 7 f7 250 non-null int64 8 f8 250 non-null int64 9 f9 250 non-null int64 10 l 250 non-null int64 dtypes: int64(11) memory usage: 21.6 KB

Defines column names for the features data Puts the features data into a DataFrame object Puts the labels data into the same DataFrame object Shows the meta information for the data set Two major problems can be identified given the results from executing the following Python code. First, not all patterns are in the sample data set. Second, the sample size is much too small per observed pattern. Even without digging deeper, it is clear that no classification algorithm can really learn about all the possible patterns in a meaningful way: In [63]: grouped = data.groupby(list(data.columns)) In [64]: freq = grouped['l'].size().unstack(fill_value=0) In [65]: freq['sum'] = freq[0] + freq[1] In [66]: freq.head(10) Out[66]: l f0 f1 f2 f3 f4 f5 f6 f7 f8 f9 0 0 0 0 0 0 0 1 1 1 1 0 1 0 1 1 0 0 0 0 1 1 1 1 1 0 0 0

24

|

Chapter 1: Artificial Intelligence

0

1

sum

0 1 0 1 0 0 0

1 1 1 0 1 1 1

1 2 1 1 1 1 1

1

0

0 1

1 0 0 1 1 1 0 0

0 1 1 0 1 0

1 1 1

In [67]: freq['sum'].describe().astype(int) Out[67]: count 227 mean 1 std 0 min 1 25% 1 50% 1 75% 1 max 2 Name: sum, dtype: int64

Groups the data along all columns Unstacks the grouped data for the labels column Adds up the frequency for a 0 and a 1 Shows the frequencies for a 0 and a 1 given a certain pattern Provides statistics for the sum of the frequencies The following Python code uses the MLPClassifier model from scikit-learn.8 The model is trained on the whole data set. What about the ability of a neural network to learn about the relationships within a given data set? The ability is pretty high, as the in-sample accuracy score shows. It is in fact close to 100%, a result driven to a large extent by the relatively high neural network capacity given the relatively small data set: In [68]: from sklearn.neural_network import MLPClassifier from sklearn.metrics import accuracy_score In [69]: model = MLPClassifier(hidden_layer_sizes=[128, 128, 128], max_iter=1000, random_state=100) In [70]: model.fit(data[fcols], data['l']) Out[70]: MLPClassifier(hidden_layer_sizes=[128, 128, 128], max_iter=1000, random_state=100) In [71]: accuracy_score(data['l'], model.predict(data[fcols])) Out[71]: 0.952

8 For details, see sklearn.neural_network.MLPClassifier.

Importance of Data

|

25

But what about the predictive power of a trained neural network? To this end, the given data set can be split into a training and a test data sub-set. The model is trained on the training data sub-set only and then tested with regard to its predictive power on the test data set. As before, the accuracy of the trained neural network is pretty high in-sample (that is, on the training data set). However, it is more than 10 percent‐ age points worse than an uninformed baseline algorithm on the test data set: In [72]: split = int(len(data) * 0.7) In [73]: train = data[:split] test = data[split:] In [74]: model.fit(train[fcols], train['l']) Out[74]: MLPClassifier(hidden_layer_sizes=[128, 128, 128], max_iter=1000, random_state=100) In [75]: accuracy_score(train['l'], model.predict(train[fcols])) Out[75]: 0.9714285714285714 In [76]: accuracy_score(test['l'], model.predict(test[fcols])) Out[76]: 0.38666666666666666

Splits the data into train and test data sub-sets Trains the model on the training data set only Reports the accuracy in-sample (training data set) Reports the accuracy out-of-sample (test data set) Roughly speaking, the neural network, trained on a small data set only, learns wrong relationships due to the identified two major problem areas. The problems are not really relevant in the context of learning relationships in-sample. To the contrary, the smaller a data set is, the more easily in-sample relationships can be learned in general. However, the problem areas are highly relevant when using the trained neural net‐ work to generate predictions out-of-sample.

Larger Data Set Fortunately, there is often a clear way out of this problematic situation: a larger data set. Faced with real-world problems, this theoretical insight might be equally correct. From a practical point of view, though, such larger data sets are not always available, nor can they often be generated so easily. However, in the context of the example of this section, a larger data set is indeed easily created. The following Python code increases the number of samples in the initial sample data set significantly. The result is that the prediction accuracy of the trained neural

26

|

Chapter 1: Artificial Intelligence

network increases by more than 10 percentage points, to a level of about 50%, which is to be expected given the nature of the labels data. It is now in line with an unin‐ formed baseline algorithm: In [77]: factor = 50 In [78]: big = pd.DataFrame(np.random.randint(0, 2, (factor * n, f)), columns=fcols) In [79]: big['l'] = np.random.randint(0, 2, factor * n) In [80]: train = big[:split] test = big[split:] In [81]: model.fit(train[fcols], train['l']) Out[81]: MLPClassifier(hidden_layer_sizes=[128, 128, 128], max_iter=1000, random_state=100) In [82]: accuracy_score(train['l'], model.predict(train[fcols])) Out[82]: 0.9657142857142857 In [83]: accuracy_score(test['l'], model.predict(test[fcols])) Out[83]: 0.5043407707910751

Prediction accuracy in-sample (training data set) Prediction accuracy out-of-sample (test data set) A quick analysis of the available data, as shown next, explains the increase in the pre‐ diction accuracy. First, all possible patterns are now represented in the data set. Sec‐ ond, all patterns have an average frequency of above 10 in the data set. In other words, the neural network sees basically all the patterns multiple times. This allows the neural network to “learn” that both labels 0 and 1 are equally likely for all possible patterns. Of course, it is a rather involved way of learning this, but it is a good illus‐ tration of the fact that a relatively small data set might often be too small in the context of neural networks: In [84]: grouped = big.groupby(list(data.columns)) In [85]: freq = grouped['l'].size().unstack(fill_value=0) In [86]: freq['sum'] = freq[0] + freq[1] In [87]: freq.head(6) Out[87]: l f0 f1 f2 f3 f4 f5 f6 f7 f8 f9 0 0 0 0 0 0 0 0 0 0 1 1 0 1

0

1

sum

10 5 2 6

9 4 5 6

19 9 7 12

Importance of Data

|

27

1 0

0 1

9 8 7 4

17 11

In [88]: freq['sum'].describe().astype(int) Out[88]: count 1024 mean 12 std 3 min 2 25% 10 50% 12 75% 15 max 26 Name: sum, dtype: int64

Adds the frequency for the 0 and 1 values Shows summary statistics for the sum values

Volume and Variety In the context of neural networks that perform prediction tasks, the volume and variety of the available data used to train the neural network are decisive for its prediction performance. The numeri‐ cal, hypothetical examples in this section show that the same neural network trained on a relatively small and not-as-varied data set underperforms its counterpart trained on a relatively large and var‐ ied data set by more than 10 percentage points. This difference can be considered huge given that AI practitioners and companies often fight for improvements as small as a tenth of a percentage point.

Big Data What is the difference between a larger data set and a big data set? The term big data has been used for more than a decade now to mean a number of things. For the pur‐ poses of this book, one might say that a big data set is large enough—in terms of vol‐ ume, variety, and also maybe velocity—for an AI algorithm to be trained properly such that the algorithm performs better at a prediction task as compared to a baseline algorithm. The larger data set used before is still small in practical terms. However, it is large enough to accomplish the specified goal. The required volume and variety of the data set are mainly driven by the structure and characteristics of the features and labels data. In this context, assume that a retail bank implements a neural network–based classifi‐ cation approach for credit scoring. Given in-house data, the responsible data scientist

28

|

Chapter 1: Artificial Intelligence

designs 25 categorical features, every one of which can take on 8 different values. The resulting number of patterns is astronomically large: In [89]: 8 ** 25 Out[89]: 37778931862957161709568

It is clear that no single data set can provide a neural network with exposure to every single one of these patterns.9 Fortunately, in practice this is not necessary for the neu‐ ral network to learn about the creditworthiness based on data for regular, defaulting, and/or rejected debtors. It is also not necessary in general to generate “good” predic‐ tions with regard to the creditworthiness of every potential debtor. This is due to a number of reasons. To name only a few, first, not every pattern will be relevant in practice—some patterns might simply not exist, might be impossible, and so forth. Second, not all features might be equally important, reducing the number of relevant features and thereby the number of possible patterns. Third, a value of 4 or 5 for feature number 7, say, might not make a difference at all, further reducing the number of relevant patterns.

Conclusions For this book, artificial intelligence, or AI, encompasses methods, techniques, algo‐ rithms, and so on that are able to learn relationships, rules, probabilities, and more from data. The focus lies on supervised learning algorithms, such as those for estima‐ tion and classification. With regard to algorithms, neural networks and deep learning approaches are at the core. The central theme of this book is the application of neural networks to one of the core problems in finance: the prediction of future market movements. More specifi‐ cally, the problem might be to predict the direction of movement for a stock index or the exchange rate for a currency pair. The prediction of the future market direction (that is, whether a target level or price goes up or down) is a problem that can be easily cast into a classification setting. Before diving deeper into the core theme itself, the next chapter first discusses selected topics related to what is called superintelligence and technological singularity. That discussion will provide useful background for the chapters that follow, which focus on finance and the application of AI to the financial domain.

9 Nor would current compute technology allow one to model and train a neural network based on such a data

set if it would be available. In this context, the next chapter discusses the importance of hardware for AI.

Conclusions

|

29

References Books and papers cited in this chapter: Alpaydin, Ethem. 2016. Machine Learning. MIT Press, Cambridge. Chollet, Francois. 2017. Deep Learning with Python. Shelter Island: Manning. Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. Cam‐ bridge: MIT Press. http://deeplearningbook.org. Kratsios, Anastasis. 2019. “Universal Approximation Theorems.” https://oreil.ly/ COOdI. Silver, David et al. 2016. “Mastering the Game of Go with Deep Neural Networks and Tree Search.” Nature 529 (January): 484-489. Shanahan, Murray. 2015. The Technological Singularity. Cambridge: MIT Press. Tegmark, Max. 2017. Life 3.0: Being Human in the Age of Artificial Intelligence. United Kingdom: Penguin Random House. VanderPlas, Jake. 2017. Python Data Science Handbook. Sebastopol: O’Reilly.

30

|

Chapter 1: Artificial Intelligence

CHAPTER 2

Superintelligence

The fact that there are many paths that lead to superintelligence should increase our confidence that we will eventually get there. If one path turns out to be blocked, we can still progress. —Nick Bostrom (2014)

There are multiple definitions for the term technological singularity. Its use dates back at least to the article by Vinge (1993), which the author provocatively begins like this: Within thirty years, we will have the technological means to create superhuman intelli‐ gence. Shortly after, the human era will be ended.

For the purposes of this chapter and book, technological singularity refers to a point in time at which certain machines achieve superhuman intelligence, or superintelligence —this is mostly in line with the original idea of Vinge (1993). The idea and concept was further popularized by the widely read and cited book by Kurzweil (2005). Barrat (2013) has a wealth of historical and anecdotal information around the topic. Shana‐ han (2015) provides an informal introduction and overview of its central aspects. The expression technological singularity itself has its origin in the concept of a singularity in physics. It refers to the center of a black hole, where mass is highly concentrated, gravitation becomes infinite, and traditional laws of physics break down. The begin‐ ning of the universe, the so-called Big Bang, is also referred to as a singularity. Although the general ideas and concepts of the technological singularity and of superintelligence might not have an obvious and direct relationship to AI applied to finance, a better understanding of their background, related problems, and potential consequences is beneficial. The insights gained in the general framework are impor‐ tant in a narrower context as well, such as for AI in finance. Those insights also help guide the discussion about how AI might reshape the financial industry in the near and long term.

31

“Success Stories” on page 32 takes a look at a selection of recent success stories in the field of AI. Among others, it covers how the company DeepMind solved the problem of playing Atari 2600 games with neural networks. It also tells the story of how the same company solved the problem of playing the game of Go at above-human-expert level. The story of chess and computer programs is also recounted in that section. “Importance of Hardware” on page 42 discusses the importance of hardware in the context of these recent success stories. “Forms of Intelligence” on page 44 introduces different forms of intelligence, such as artificial narrow intelligence (ANI), artificial general intelligence (AGI), and superintelligence (SI). “Paths to Superintelligence” on page 45 is about potential paths to superintelligence, such as whole brain emulation (WBE), while “Intelligence Explosion” on page 50 is about what researchers call intel‐ ligence explosion. “Goals and Control” on page 50 provides a discussion of aspects related to the so-called control problem in the context of superintelligence. Finally, “Potential Outcomes” on page 54 briefly looks at potential future outcomes and scenarios once superintelligence has been achieved.

Success Stories Many ideas and algorithms in AI date back a few decades already. Over these decades there have been longer periods of hope on the one hand and despair on the other hand. Bostrom (2014, ch. 1) provides a review of these periods. In 2020, one can say for sure that AI is in the middle of a period of hope, if not excite‐ ment. One reason for this is recent successes in applying AI to domains and problems that even a few years ago seemed immune to AI dominance for decades to come. The list of such success stories is long and growing rapidly. Therefore, this section focuses on three such stories only. Gerrish (2018) provides a broader selection and more detailed accounts of the single cases.

Atari This sub-section first tells the success story of how DeepMind mastered playing Atari 2600 games with reinforcement learning and neural networks, and then illustrates the basic approach that led to its success based on a concrete code example.

The story The first success story is about playing Atari 2600 games on a superhuman level.1 The Atari 2600 Video Computer System (VCS) was released in 1977 and was one of the first widespread game-playing consoles in the 1980s. Selected popular games from

1 For background and historical information, see http://bit.ly/aiif_atari.

32

|

Chapter 2: Superintelligence

that period, such as Space Invaders, Asteroids, or Missile Command, count as classics and are still played decades later by retro games enthusiasts. DeepMind published a paper (Mnih et al. 2013) in which its team detailed results from applying reinforcement learning to the problem of playing Atari 2600 games by an AI algorithm, or a so-called AI agent. The algorithm is a variant of Q-learning applied to a convolutional neural network.2 The algorithm is trained on highdimensional visual input (raw pixels) only, without any guidance by or input from a human. The original project focused on seven Atari 2600 games, and for three of them—Pong, Enduro, and Breakout—the DeepMind team reported above-human expert performance of the AI agent. From an AI point of view, it is remarkable not only that the DeepMind team achieved such a result, but also how it achieved it. First, the team only used a single neural net‐ work to learn and play all seven games. Second, no human guidance or humanly labeled data was provided, just the interactive learning experience based on visual input properly transformed into features data.3 Third, the approach used is reinforce‐ ment learning, which relies on observation of the relationships between actions and outcomes (rewards) only—basically the same way a human player learns to play such a game. One of the Atari 2600 games, for which the DeepMind AI agent achieved abovehuman expert performance, is Breakout. In this game, the goal is to destroy lines of bricks at the top of the screen by using a paddle at the bottom of the screen from which a ball bounces back and moves straight across the screen. Whenever the ball hits a brick, the brick is destroyed and the ball bounces back. The ball also bounces back from the left, right, and top walls. The player loses a life in this game whenever the ball reaches the bottom of the screen without being hit by the paddle. The action space has three elements, all related to the paddle: staying at current posi‐ tion, moving left, and moving right. The state space is represented by frames of the game screen of size 210 x 160 pixels with a 128-color palette. The reward is repre‐ sented by the game score, which the DeepMind algorithm is programmed to maxi‐ mize. With regard to the action policy, the algorithm learns which action is best to take, given a certain game state, to maximize the game score (total reward).

An example There is not enough room in this chapter to explore in detail the approach taken by DeepMind for Breakout and the other Atari 2600 games. However, the OpenAI Gym

2 For details, refer to Mnih et al. (2013). 3 Among other factors, this is made possible by the availability of the Arcade Learning Environment (ALE) that

allows researchers to train AI agents for Atari 2600 games via a standardized API.

Success Stories

|

33

environment (see https://gym.openai.com) allows for the illustration of a similar, but simpler, neural network approach for a similar, but again simpler, game. The Python code in this section works with the CartPole environment of the OpenAI Gym (see http://bit.ly/aiif_cartpole).4 In this environment, a cart needs to be moved to the right or left to balance an upright pole on top of the paddle. Therefore, the action space is similar to the Breakout action space. The state space consists of four physical data points: cart position, cart velocity, pole angle, and pole angular velocity (see Figure 2-1). If, after having taken an action, the pole is still in balance, the agent gets a reward of 1. If the pole is out of balance, the game ends. The agent is considered suc‐ cessful if it reaches a total reward of 200.5

Figure 2-1. Graphical representation of the CartPole environment The following code first instantiates a CartPole environment object, and then inspects the action and state spaces, takes a random action, and captures the results. The AI agent moves on toward the next round when the done variable is False: In [1]: import gym import numpy as np import pandas as pd np.random.seed(100) In [2]: env = gym.make('CartPole-v0') In [3]: env.seed(100) Out[3]: [100] In [4]: action_size = env.action_space.n action_size Out[4]: 2 In [5]: [env.action_space.sample() for _ in range(10)] Out[5]: [1, 0, 0, 0, 1, 1, 0, 0, 0, 0] In [6]: state_size = env.observation_space.shape[0] state_size Out[6]: 4

4 Chapter 9 revisits this example in more detail. 5 More specifically, an AI agent is considered successful if it reaches an average total reward of 195 or more

over 100 consecutive games.

34

| Chapter 2: Superintelligence

In [7]: state = env.reset() state # [cart position, cart velocity, pole angle, pole angular velocity] Out[7]: array([-0.01628537, 0.02379786, -0.0391981 , -0.01476447]) In [8]: state, reward, done, _ = env.step(env.action_space.sample()) state, reward, done, _ Out[8]: (array([-0.01580941, -0.17074066, -0.03949338, 0.26529786]), 1.0, False, {})

Instantiates the environment object Fixes the random number seed for the environment Shows the size of the action space Takes some random actions and collects them Shows the size of the state space Resets (initializes) the environment and captures the state Takes a random action and steps the environment forward to the next state The next step is to play the game based on random actions to generate a large enough data set. However, to increase the quality of the data set, only data that results from games with a total reward of 110 or more is collected. To this end, a few thousand games are played to collect enough data for the training of a neural network: In [9]: %%time data = pd.DataFrame() state = env.reset() length = [] for run in range(25000): done = False prev_state = env.reset() treward = 1 results = [] while not done: action = env.action_space.sample() state, reward, done, _ = env.step(action) results.append({'s1': prev_state[0], 's2': prev_state[1], 's3': prev_state[2], 's4': prev_state[3], 'a': action, 'r': reward}) treward += reward if not done else 0 prev_state = state if treward >= 110: data = data.append(pd.DataFrame(results)) length.append(treward) CPU times: user 9.84 s, sys: 48.7 ms, total: 9.89 s Wall time: 9.89 s

Success Stories

|

35

In [10]: np.array(length).mean() Out[10]: 119.75 In [11]: data.info()

Int64Index: 479 entries, 0 to 143 Data columns (total 6 columns): # Column Non-Null Count Dtype --- ------ -------------- ----0 s1 479 non-null float64 1 s2 479 non-null float64 2 s3 479 non-null float64 3 s4 479 non-null float64 4 a 479 non-null int64 5 r 479 non-null float64 dtypes: float64(5), int64(1) memory usage: 26.2 KB In [12]: data.tail() Out[12]: s1 139 0.639509 140 0.659363 141 0.675345 142 0.687466 143 0.695736

s2 0.992699 0.799086 0.606042 0.413513 0.610658

s3 -0.112029 -0.143006 -0.168869 -0.189837 -0.206100

s4 a -1.548863 0 -1.293131 0 -1.048421 0 -0.813148 1 -1.159030 0

r 1.0 1.0 1.0 1.0 1.0

Only if the total reward of the random agent is at least 100… …is the data is collected… …and the total reward recorded. The average total reward of all random games included in the data set. A look at the collected data in the DataFrame object. Equipped with the data, a neural network can be trained as follows. Set up a neural network for classification. Train it with the columns representing the state data as fea‐ tures and the column with the taken actions as labels data. Given that the data set only includes actions that have been successful for the given state, the neural network learns about what action to take (label) given the state (features): In [13]: from pylab import plt plt.style.use('seaborn') %matplotlib inline In [14]: import tensorflow as tf tf.random.set_seed(100) In [15]: from keras.layers import Dense

36

|

Chapter 2: Superintelligence

from keras.models import Sequential Using TensorFlow backend. In [16]: model = Sequential() model.add(Dense(64, activation='relu', input_dim=env.observation_space.shape[0])) model.add(Dense(1, activation='sigmoid')) model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['acc']) In [17]: %%time model.fit(data[['s1', 's2', 's3', 's4']], data['a'], epochs=25, verbose=False, validation_split=0.2) CPU times: user 1.02 s, sys: 166 ms, total: 1.18 s Wall time: 797 ms Out[17]: In [18]: res = pd.DataFrame(model.history.history) res.tail(3) Out[18]: val_loss val_acc loss acc 22 0.660300 0.59375 0.646965 0.626632 23 0.660828 0.59375 0.646794 0.621410 24 0.659114 0.59375 0.645908 0.626632

A neural network with one hidden layer only is used. The model is trained on the previously collected data. The metrics per training step are shown for the final few steps. The trained neural network, or AI agent, can then play the CartPole game given its learned best actions for any state it is presented with. The AI agent achieves the maximum total reward of 200 for each of the 100 games played. This is based on a relatively small data set in combination with a relatively simple neural network: In [20]: def epoch(): done = False state = env.reset() treward = 1 while not done: action = np.where(model.predict(np.atleast_2d(state))[0][0] > \ 0.5, 1, 0) state, reward, done, _ = env.step(action) treward += reward if not done else 0 return treward In [21]: res = np.array([epoch() for _ in range(100)]) res Out[21]: array([200., 200., 200., 200., 200., 200., 200., 200., 200., 200., 200.,

Success Stories

|

37

200., 200., 200., 200., 200., 200., 200., 200., 200., 200., 200., 200., 200., 200., 200., 200., 200.])

200., 200., 200., 200., 200., 200., 200., 200.,

200., 200., 200., 200., 200., 200., 200., 200.,

200., 200., 200., 200., 200., 200., 200., 200.,

200., 200., 200., 200., 200., 200., 200., 200.,

200., 200., 200., 200., 200., 200., 200., 200.,

200., 200., 200., 200., 200., 200., 200., 200.,

200., 200., 200., 200., 200., 200., 200., 200.,

200., 200., 200., 200., 200., 200., 200., 200.,

200., 200., 200., 200., 200., 200., 200., 200.,

In [22]: res.mean() Out[22]: 200.0

Chooses an action given the state and the trained model Moves the environment one step forward based on the learned action Plays a number of games and records the total reward for each game Calculates the average total reward for all games The Arcade Learning Environment (ALE) works similarly to OpenAI Gym. It allows one to programmatically interact with emulated Atari 2600 games, take actions, col‐ lect the results from a taken action, and so on. The task of learning to play Breakout, for example, is of course more involved, if only because the state space is much larger. The basic approach, however, is similar to the one taken here, with several algorith‐ mic refinements.

Go The board game Go is more than 2,000 years old. It has long been considered a cre‐ ation of beauty and art—because it is simple in principle but nevertheless highly com‐ plex—and was expected to withstand the advance of game-playing AI agents for decades to come. The strength of a Go player is measured in dans, in line with gradu‐ ation systems for many martial arts systems. For example, Lee Sedol, who was the Go world champion for years, holds the 9th dan. In 2014, Bostrom postulated: Go-playing programs have been improving at a rate of about 1 dan/year in recent years. If this rate of improvement continues, they might beat the human world cham‐ pion in about a decade.

Again, it was a team at DeepMind that was able to achieve breakthroughs for AI agents playing Go with its AlphaGo algorithm (see the AlphaGo page on DeepMind’s website). In Silver et al. (2016), the researchers describe the situation as follows: The game of Go has long been viewed as the most challenging of classic games for arti‐ ficial intelligence owing to its enormous search space and the difficulty of evaluating board positions and moves.

38

|

Chapter 2: Superintelligence

The members of the team used a combination of a neural network with a Monte Carlo tree search algorithm, which they briefly sketch in their paper. Recounting their early successes from 2015, the team points out in the introduction: [O]ur program AlphaGo achieved a 99.8% winning rate against other Go programs, and defeated the human European Go champion [Fan Hui] by 5 games to 0. This is the first time that a computer program has defeated a human professional player in the full-sized game of Go, a feat previously thought to be at least a decade away.

It is remarkable that this milestone was achieved just one year after a leading AI researcher, Nick Bostrom, predicted that it might take another decade to reach that level. Many observers remarked, however, that the beating European Go champion of that time, Fan Hui, cannot really be considered a benchmark since the world Go elite play on a much higher level. The DeepMind team took on the challenge and organ‐ ized in March 2016 a best-of-five-games competition against the then 18-time world Go champion Lee Sedol—for sure a proper benchmark for elite-level human Go play‐ ing. (A wealth of background information is provided on the AlphaGo Korea web page, and there is even a movie available about the event.) To this end, the DeepMind team further improved the AlphaGo Fan version to the AlphaGo Lee iteration. The story of the competition and AlphaGo Lee is well documented and has drawn attention all over the world. DeepMind writes on its web page: AlphaGo’s 4-1 victory in Seoul, South Korea, on March 2016 was watched by over 200 million people worldwide. This landmark achievement was a decade ahead of its time. The game earned AlphaGo a 9 dan professional ranking, the highest certification. This was the first time a computer Go player had ever received the accolade.

Up until that point, AlphaGo used, among other resources, training data sets based on millions of human expert games for its supervised learning. The team’s next itera‐ tion, AlphaGo Zero, skipped that approach completely and relied instead on rein‐ forcement learning and self-play only, putting together different generations of trained, neural network–based AI agents to compete against each other. Silver et al.’s article (2017b) provides details of AlphaGo Zero. In the abstract, the researchers summarize: AlphaGo becomes its own teacher: a neural network is trained to predict AlphaGo’s own move selections and also the winner of AlphaGo’s games. This neural network improves the strength of the tree search, resulting in higher quality move selection and stronger self-play in the next iteration. Starting tabula rasa, our new program AlphaGo Zero achieved superhuman performance, winning 100–0 against the previously pub‐ lished, champion-defeating AlphaGo.

It is remarkable that a neural network trained not too dissimilarly to the CartPole example from the previous section (that is, based on self-play) can crack a game as complex as Go, whose possible board positions outnumber the atoms of the universe. It is also remarkable that the Go wisdom collected over centuries by human players is simply not necessary to achieve this milestone. Success Stories

|

39

The DeepMind team did not stop there. AlphaZero was intended to be a general game-playing AI agent that was supposed to be able to learn different complex board games, such as Go, chess, and shogi. With regard to AlphaZero, the team summarizes in Silver (2017a): In this paper, we generalise this approach into a single AlphaZero algorithm that can achieve, tabula rasa, superhuman performance in many challenging domains. Starting from random play, and given no domain knowledge except the game rules, AlphaZero achieved within 24 hours a superhuman level of play in the games of chess and shogi (Japanese chess) as well as Go, and convincingly defeated a world-champion program in each case.

Again, a remarkable milestone was reached by DeepMind in 2017: a game-playing AI agent that, after less than 24 hours of self-playing and training, achieved abovehuman-expert levels in three intensely studied board games with centuries-long histories in each case.

Chess Chess is, of course, one of the most popular board games in the world. Chess-playing computer programs have been around since the very early days of computing, and in particular, home computing. For example, an almost complete chess engine called ZX Chess, which only consisted of about 672 bytes of machine code, was introduced in 1983 for the ZX-81 Spectrum home computer.6 Although an incomplete implementa‐ tion that lacked certain rules like castling, it was a great achievement at the time and is still fascinating for computer chess fans today. The record of ZX Chess as the small‐ est chess program stood for 32 years and was broken only by BootChess in 2015, at 487 bytes.7 It can almost be considered software engineering genius to write a computer program with such a small code base that can play a board game that has more possible per‐ mutations of a game than the universe has atoms. While not being as complex with regard to the pure numbers as Go, chess can be considered one of the most challeng‐ ing board games, as players take decades to reach grandmaster level. In the mid-1980s, expert-level computer chess programs were still far away, even on better hardware with many fewer constraints than the basic home computer ZX-81 Spectrum. No wonder then that leading chess players at that time felt confident when playing against computers. For example, Garry Kasparov (2017) recalls an event in 1985 during which he played 32 simultaneous games as follows:

6 See http://bit.ly/aiif_1k_chess for an electronic reprint of the original article published in the February 1983

issue of Your Computer and scans of the original code.

7 See http://bit.ly/aiif_bootchess for more background.

40

| Chapter 2: Superintelligence

It was a pleasant day in Hamburg in June 6, 1985….Each of my opponents, all thirtytwo of them, was a computer…it didn’t come as much of a surprise…when I achieved a perfect 32-0 score.

It took computer chess developers and the hardware experts from International Busi‐ ness Machines Corporation (IBM) 12 years until a computer called Deep Blue was able to beat Kasparov, then the human world chess champion. In his book, published 20 years after his historic loss against Deep Blue, he writes: Twelve years later I was in New York City fighting for my chess life. Against just one machine, a $10 million IBM supercomputer nicknamed “Deep Blue.”

Kasparov played a total of six games against Deep Blue. The computer won with 3.5 points to Kasparov’s 2.5; whereby a full point is awarded for a win and half a point to each player for a draw. While Deep Blue lost the first game, it would win two of the remaining five, with three games ending in a draw by mutual agreement. It has been pointed out that Deep Blue should not be considered a form of AI since it mainly relied on a huge hardware cluster. This hardware cluster with 30 nodes and 480 special-purpose chess chips—designed by IBM specifically for this event—could ana‐ lyze some 200 million positions per second. In that sense, Deep Blue mainly relied on brute force techniques rather than modern AI algorithms such as neural networks. Since 1997, both hardware and software have seen tremendous advancements. Kas‐ parov sums it up as follows when he refers in his book to chess applications on modern smartphones: Jump forward another 20 years to today, to 2017, and you can download any number of free chess apps for your phone that rival any human grandmaster.

The hardware requirements to beat a human grandmaster have fallen from $10 mil‐ lion to about $100 (that is, by a factor of 100,000). However, chess applications for regular computers and smartphones still rely on the collected wisdom of decades of computer chess. They embody a large number of human-designed rules and strate‐ gies for the game, rely on a large database for openings, and then benefit from the increased compute power and memory of modern devices for their mostly brute force–based evaluation of millions of chess positions. This is where AlphaZero comes in. The approach of AlphaZero to mastering the game of chess is exclusively based on reinforcement learning with self-play of different versions of the AI agent competing against each other. The DeepMind team contrasts the traditional approach to computer chess with AlphaZero as follows (see AlphaZero research paper): Traditional chess engines, including the world computer chess champion Stock‐ fish and IBM’s ground-breaking Deep Blue, rely on thousands of rules and heuristics handcrafted by strong human players that try to account for every eventuality in a game….AlphaZero takes a totally different approach, replacing these hand-crafted

Success Stories

|

41

rules with a deep neural network and general purpose algorithms that know nothing about the game beyond the basic rules.

Given this tabula rasa approach of AlphaZero, its performance after a few hours of self-play-based training is exceptional when compared to the leading traditional chess-playing computer programs. AlphaZero only needs nine hours or less of train‐ ing to master chess on a level that surpasses every human player and every other computer chess program, including the Stockfish engine, which at one time domina‐ ted computer chess. In a 2016 test series comprising 1,000 games, AlphaZero beat Stockfish by winning 155 games (mostly while playing white), losing just six games, and drawing the rest. While IBM’s Deep Blue was able to analyze 200 million positions per second, modern chess engines, such as Stockfish, on many-core commodity hardware, can analyze some 60 million positions per second. At the same time, AlphaZero only analyzes about 60,000 positions per second. Despite analyzing 1,000 times fewer positions per second, it nevertheless is able to beat Stockfish. One might be inclined to think that AlphaZero indeed shows some form of intelligence that sheer brute force cannot compensate for. Given that human grandmasters can maybe analyze a few hundred positions per second based on experience, patterns, and intuition, AlphaZero might inhabit a sweet spot between expert human chess player and traditional chess engine based on a brute-force approach, aided by handcrafted rules and stored chess knowl‐ edge. One could speculate that AlphaZero acquires something similar to human pat‐ tern recognition, foresight, and intuition combined with higher computational speeds due to its comparatively better hardware for that purpose.

Importance of Hardware AI researchers and practitioners have made tremendous progress over the past dec‐ ade with regard to AI algorithms. Reinforcement learning, generally combined with neural networks for action policy representation, has proven useful and superior in many different areas, as the previous section illustrates. However, without advances on the hardware side, the recent AI achievements would not have been possible. Again, the story of DeepMind and its effort to master the game of Go with reinforcement learning (RL) provides some valuable insights. Table 2-1 provides an overview of the hardware usage and power consumption for the major AlphaGo versions from 2015 onwards.8 Not only has the strength of

8 For more on this, see: https://oreil.ly/im174.

42

|

Chapter 2: Superintelligence

AlphaGo increased steadily, but both the hardware requirements and the associated power consumption have also come down dramatically.9 Table 2-1. DeepMind hardware for AlphaGo Version AlphaGo Fan

Year 2015

Elo ratinga >3,000

Hardware 176 GPUs

Power consumption [TDP] >40,000

AlphaGo Lee

2016

>3,500

48 TPUs

10,000+

AlphaGo Master

2016

>4,500

4 TPUs

5,000

4 TPUs

α2. Expected Utility Theory

|

67

Utility functions A utility function is a way to represent the preferences ⪰ of an agent in a mathemati‐ cal and numerical way in that such a function assigns a numerical value to a certain payoff. In this context, the absolute value is not relevant. It is rather the ordering that the values induce that is of interest.6 Assume that � represents all possible payoffs over which the agent can express her preferences. A utility function U is then defined as follows: U:�

ℝ+, x

U x

If U represents the preferences ⪰ of an agent, then the following relationships hold true: A≻B A⪰B A≺B A⪯B A∼B

U U U U U

A A A A A

>U ≥U 0 does so as well. Regarding utility functions, von Neumann and Morgenstern (1944, p. 25) summarize as follows: “So we see: If such a numerical valuation of utilities exists at all, then it is determined up to a linear transformation. I.e. then utility is a number up to a linear transformation.”

Expected utility functions Von Neumann and Morgenstern (1944) show that if the preferences of an agent ⪰ satisfy the preceding five axioms, then there exists an expected utility function of the form: U:�

ℝ+, x

P

� ux =

Ω

∑ω P ω u x ω

Here, u: ℝ ℝ, x u x is a monotonically increasing, state-independent function, often called Bernoulli utility, such as u x = ln x , u x = x, or u x = x2.

6 One speaks in general of ordinal numbers. House numbers in streets are a good example for ordinal numbers.

68

|

Chapter 3: Normative Finance

In words, the expected utility function U first applies a function u to the payoff x ω in a certain state and then uses the probabilities for a given state to occur P ω to weigh the single utilities. In the special case of linear Bernoulli utility u x = x, the expected utility is simply the expected value of the state-dependent payoff, U x = �P x .

Risk aversion In finance, the concept of risk aversion is important. The most commonly used meas‐ ure of risk aversion is the Arrow-Pratt measure of absolute risk aversion (ARA), which dates back to Pratt (1964). Assume that an agent’s state-independent Bernoulli utility function is u x . Then the Arrow-Pratt measure of ARA is defined by the following: ARA x = −

u′′ x ,x ≥ 0 u′ x

The following three cases can be distinguished according to this measure: u′′ x ARA x = − u′ x

> 0 risk‐averse = 0 risk‐neutral < 0 risk‐loving

In financial theories and models, risk aversion and risk neutrality in general are assumed to be appropriate cases. In gambling, risk-loving agents probably can be found as well. Consider the three Bernoulli functions previously mentioned: u x = ln x , u x = x, or u x = x2. It is easily verified that they model risk-averse, risk-neutral, and riskloving agents, respectively. Consider, for example, u x = x2: −

u′′ x 2 = − < 0, x > 0 2x u′ x

risk‐loving

Numerical Example The application of EUT is easily illustrated in Python. Assume the example model economy from the previous section ℳ 2. Assume that an agent with preferences ⪰ decides according to the EUT between different future payoffs. The Bernoulli utility of the agent is given by u x = x. In the example, payoff A1 resulting from portfolio ϕA is preferred over the payoff D1 resulting from portfolio ϕD.

Expected Utility Theory

|

69

The following code illustrates this application: In [11]: def u(x): return np.sqrt(x) In [12]: phi_A = np.array((0.75, 0.25)) phi_D = np.array((0.25, 0.75)) In [13]: np.dot(M0, phi_A) == np.dot(M0, phi_D) Out[13]: True In [14]: A1 = np.dot(M1, phi_A) A1 Out[14]: array([17.75, 6.5 ]) In [15]: D1 = np.dot(M1, phi_D) D1 Out[15]: array([13.25, 9.5 ]) In [16]: P = np.array((0.5, 0.5)) In [17]: def EUT(x): return np.dot(P, u(x)) In [18]: EUT(A1) Out[18]: 3.381292321692286 In [19]: EUT(D1) Out[19]: 3.3611309730623735

The risk-averse Bernoulli utility function Two portfolios with different weights Shows that the cost to set up each portfolio is the same The uncertain payoff of one portfolio… …and the other one The probability measure The expected utility function The utility values for the two uncertain payoffs A typical problem in this context is to derive an optimal portfolio (that is, one that maximizes the expected utility) given a fixed budget of the agent w > 0. The following Python code models this problem and solves it exactly. Of its available budget, the

70

|

Chapter 3: Normative Finance

agent puts roughly 60% in the risky asset and roughly 40% in the risk-less asset. The results are mainly driven by the particular form of the Bernoulli utility function: In [20]: from scipy.optimize import minimize In [21]: w = 10 In [22]: cons = {'type': 'eq', 'fun': lambda phi: np.dot(M0, phi) - w} In [23]: def EUT_(phi): x = np.dot(M1, phi) return EUT(x) In [24]: opt = minimize(lambda phi: -EUT_(phi), x0=phi_A, constraints=cons) In [25]: opt Out[25]:

fun: jac: message: nfev: nit: njev: status: success: x:

-3.385015999493397 array([-1.69249132, -1.69253424]) 'Optimization terminated successfully.' 16 4 4 0 True array([0.61122474, 0.38877526])

In [26]: EUT_(opt['x']) Out[26]: 3.385015999493397

The fixed budget of the agent The budget constraint for use with minimize7 The expected utility function defined over portfolios Minimizing -EUT_(phi) maximizes EUT_(phi) The initial guess for the optimization The budget constraint applied The optimal results, including the optimal portfolio under x The optimal (highest) expected utility given the budget of w = 10 7 For details, see http://bit.ly/aiif_minimize.

Expected Utility Theory

|

71

Mean-Variance Portfolio Theory Mean-variance portfolio (MVP) theory, according to Markowitz (1952), is another cornerstone in financial theory. It is one of the first theories of investment under uncertainty that focused on statistical measures only for the construction of stock investment portfolios. MVP completely abstracts from, say, fundamentals of a company that might drive its stock performance or assumptions about the future competitiveness of a company that might be important for the growth prospects of a company. Basically, the only input data that counts is the time series of share prices and statistics derived therefrom, such as the (historical) annualized mean return and the (historical) annualized variance of the returns.

Assumptions and Results The central assumption of MVP, according to Markowitz (1952), is that investors only care about expected returns and the variance of these returns: We next consider the rule that the investor does (or should) consider expected return a desirable thing and variance of return an undesirable thing. This rule has many sound points, both as a maxim for, and hypothesis about, investment behavior. The portfolio with maximum expected return is not necessarily the one with minimum variance. There is a rate at which the investor can gain expected return by taking on variance, or reduce variance by giving up expected return.

This approach to investors’ preferences is quite different to the approach that defines an agent’s preferences and utility function over payoffs directly. MVP rather assumes that an agent’s preferences and utility function can be defined over the first and sec‐ ond moment of the returns an investment portfolio is expected to yield.

Implicitly Assumed Normal Distribution In general, MVP theory, focusing on one period portfolio risk and return only, is not compatible with standard EUT. One way of resolving this issue is to assume that returns of risky assets are nor‐ mally distributed such that the first and second moments are suffi‐ cient to describe the full distribution of an asset’s returns. This is something almost never observed in real financial data, as the next chapter illustrates. The other way is to assume a particular quad‐ ratic Bernoulli utility function, as shown in the next section.

Portfolio statistics Assume a static economy ℳ N = Ω, ℱ , P , � , for which the set of tradable assets � consists of N risky assets, A1, A2, ..., AN . With An0 being the fixed price of asset n today

72

| Chapter 3: Normative Finance

and An1 being its payoff in one year, the (simple) net returns vector rn of asset n is defined by the following: n

r =

An1

An0

−1

For all future states having the same probability to unfold, the expected return of asset n is given by: 1 Ω

μn =

Ω

∑ω rn ω

Accordingly, the vector of expected returns is given by the following: μ1 μ=

μ2 ⋮ μN T

n A portfolio (vector) ϕ = ϕ1, ϕ2, ..., ϕN , with ϕn ≥ 0 and ∑N n ϕ = 1, assigns weights to 8 each asset in the portfolio.

The expected return of the portfolio is then given by the dot product of the portfolio weights vector and the vector of expected returns: μ phi = ϕ · μ

Now define the covariance between assets n and m by the following: σ mn =

Ω

∑ω

rm ω − μm rn ω − μn

8 These assumptions are not really necessary. Short sales, for instance, might be allowed without altering the

analysis significantly.

Mean-Variance Portfolio Theory

|

73

The covariance matrix then is given by: σ 11 σ 12 ... σ 1n Σ=

σ 21 σ 22 ... σ 2n ⋮ ⋮ ⋱ ⋮ σ n1 σ n2 ... σ nn

The expected variance of the portfolio is then in turn given by the double dot product: φ phi = ϕT · Σ · ϕ

The expected volatility of the portfolio accordingly is the following: σ phi = φ phi

Sharpe ratio Sharpe (1966) introduces a measure to judge the risk-adjusted performance of mutual funds and other portfolios, or even single risky assets. In its simplest form, it relates the (expected, realized) return of a portfolio to its (expected, realized) volatil‐ ity. Formally, the Sharpe ratio therefore can be defined by the following: π phi =

μ phi σ phi

If r represents the risk-less short rate, the risk premium or excess return of a portfolio phi over a risk-free alternative is defined by μ phi − r. In another version of the Sharpe ratio, this risk premium is the numerator: π phi =

μ phi − r σ phi

If the risk-less short rate is relatively low, the two versions do not yield too different numerical results if the same risk-less short rate is applied. In particular, when rank‐ ing different portfolios according to the Sharpe ratio, the two versions should gener‐ ate the same ranking order, everything else equal.

74

| Chapter 3: Normative Finance

Numerical Example Getting back to the static model economy ℳ 2, the basic notions of MVP can again be easily illustrated by the use of Python.

Portfolio statistics First, here is the derivation of the portfolio expected return: In [27]: rS = S1 / S0 - 1 rS Out[27]: array([ 1. , -0.5]) In [28]: rB = B1 / B0 - 1 rB Out[28]: array([0.1, 0.1]) In [29]: def mu(rX): return np.dot(P, rX) In [30]: mu(rS) Out[30]: 0.25 In [31]: mu(rB) Out[31]: 0.10000000000000009 In [32]: rM = M1 / M0 - 1 rM Out[32]: array([[ 1. , 0.1], [-0.5, 0.1]]) In [33]: mu(rM) Out[33]: array([0.25, 0.1 ])

Return vector of the risky asset Return vector of the risk-less asset Expected return function Expected returns of the traded assets Return matrix for the traded assets Expected return vector

Mean-Variance Portfolio Theory

|

75

Second, the variance and volatility, as well as the covariance matrix: In [34]: def var(rX): return ((rX - mu(rX)) ** 2).mean() In [35]: var(rS) Out[35]: 0.5625 In [36]: var(rB) Out[36]: 0.0 In [37]: def sigma(rX): return np.sqrt(var(rX)) In [38]: sigma(rS) Out[38]: 0.75 In [39]: sigma(rB) Out[39]: 0.0 In [40]: np.cov(rM.T, aweights=P, ddof=0) Out[40]: array([[0.5625, 0. ], [0. , 0. ]])

The variance function The volatility function The covariance matrix Third, the portfolio expected return, portfolio expected variance, and portfolio expected volatility, illustrated for an equally weighted portfolio: In [41]: phi = np.array((0.5, 0.5)) In [42]: def mu_phi(phi): return np.dot(phi, mu(rM)) In [43]: mu_phi(phi) Out[43]: 0.17500000000000004 In [44]: def var_phi(phi): cv = np.cov(rM.T, aweights=P, ddof=0) return np.dot(phi, np.dot(cv, phi)) In [45]: var_phi(phi) Out[45]: 0.140625 In [46]: def sigma_phi(phi): return var_phi(phi) ** 0.5

76

|

Chapter 3: Normative Finance

In [47]: sigma_phi(phi) Out[47]: 0.375

The portfolio expected return The portfolio expected variance The portfolio expected volatility

Investment opportunity set Based on a Monte Carlo simulation for the portfolio weights ϕ, one can visualize the investment opportunity set in the volatility-return space (Figure 3-1, generated by the following code snippet).

Figure 3-1. Simulated expected portfolio volatility and return (one risky asset) Because there is only one risky asset and one risk-less asset, the opportunity set is a straight line: In [48]: from pylab import plt, mpl plt.style.use('seaborn') mpl.rcParams['savefig.dpi'] = 300 mpl.rcParams['font.family'] = 'serif' In [49]: phi_mcs = np.random.random((2, 200)) In [50]: phi_mcs = (phi_mcs / phi_mcs.sum(axis=0)).T In [51]: mcs = np.array([(sigma_phi(phi), mu_phi(phi)) for phi in phi_mcs])

Mean-Variance Portfolio Theory

|

77

In [52]: plt.figure(figsize=(10, 6)) plt.plot(mcs[:, 0], mcs[:, 1], 'ro') plt.xlabel('expected volatility') plt.ylabel('expected return');

Random portfolio compositions, normalized to 1 Expected portfolio volatility and return for the random compositions Consider now the case of a static three-state economy ℳ 3 for which Ω = u, m, d 1 1 1 holds. The three states are equally likely, P = 3 , 3 , 3 . The set of tradable assets con‐ sists of two risky assets S and T with a fixed price of S0 = T 0 = 10 and uncertain pay‐ offs, respectively, of the following: 20 S1 = 10 5

and 1 T 1 = 12 13

Based on these assumptions, the following Python code repeats the Monte Carlo sim‐ ulation and visualizes the results in Figure 3-2. With two risky assets, the well-known MVP “bullet” becomes visible. In [53]: P = np.ones(3) / 3 P Out[53]: array([0.33333333, 0.33333333, 0.33333333]) In [54]: S1 = np.array((20, 10, 5)) In [55]: T0 = 10 T1 = np.array((1, 12, 13)) In [56]: M0 = np.array((S0, T0)) M0 Out[56]: array([10, 10]) In [57]: M1 = np.array((S1, T1)).T M1 Out[57]: array([[20, 1], [10, 12], [ 5, 13]])

78

|

Chapter 3: Normative Finance

In [58]: rM = M1 / M0 - 1 rM Out[58]: array([[ 1. , -0.9], [ 0. , 0.2], [-0.5, 0.3]]) In [59]: mcs = np.array([(sigma_phi(phi), mu_phi(phi)) for phi in phi_mcs]) In [60]: plt.figure(figsize=(10, 6)) plt.plot(mcs[:, 0], mcs[:, 1], 'ro') plt.xlabel('expected volatility') plt.ylabel('expected return');

New probability measure for three states

Figure 3-2. Simulated expected portfolio volatility and return (two risky assets)

Minimum volatility and maximum Sharpe ratio Next, the derivation of the minimum volatility (minimum variance) and maximum Sharpe ratio portfolios. Figure 3-3 shows the location of the two portfolios in the riskreturn space. Although the risky asset T has a negative expected return, it has a significant weight in the maximum Sharpe ratio portfolio. This is due to diversification effects that lower the portfolio risk more than the expected return of the portfolio is reduced: In [61]: cons = {'type': 'eq', 'fun': lambda phi: np.sum(phi) - 1} In [62]: bnds = ((0, 1), (0, 1))

Mean-Variance Portfolio Theory

|

79

In [63]: min_var = minimize(sigma_phi, (0.5, 0.5), constraints=cons, bounds=bnds) In [64]: min_var Out[64]: fun: jac: message: nfev: nit: njev: status: success: x:

0.07481322946910632 array([0.07426564, 0.07528945]) 'Optimization terminated successfully.' 17 4 4 0 True array([0.46511697, 0.53488303])

In [65]: def sharpe(phi): return mu_phi(phi) / sigma_phi(phi) In [66]: max_sharpe = minimize(lambda phi: -sharpe(phi), (0.5, 0.5), constraints=cons, bounds=bnds) In [67]: max_sharpe Out[67]: fun: -0.2721654098971811 jac: array([ 0.00012054, -0.00024174]) message: 'Optimization terminated successfully.' nfev: 38 nit: 9 njev: 9 status: 0 success: True x: array([0.66731116, 0.33268884]) In [68]: plt.figure(figsize=(10, 6)) plt.plot(mcs[:, 0], mcs[:, 1], 'ro', ms=5) plt.plot(sigma_phi(min_var['x']), mu_phi(min_var['x']), '^', ms=12.5, label='minimum volatility') plt.plot(sigma_phi(max_sharpe['x']), mu_phi(max_sharpe['x']), 'v', ms=12.5, label='maximum Sharpe ratio') plt.xlabel('expected volatility') plt.ylabel('expected return') plt.legend();

Minimizes the expected portfolio volatility Defines the Sharpe ratio function, assuming a short rate of 0 Maximizes the Sharpe ratio by minimizing its negative value

80

|

Chapter 3: Normative Finance

Figure 3-3. Minimum volatility and maximum Sharpe ratio portfolios

Efficient frontier An efficient portfolio is one that has a maximum expected return (risk) given its expected risk (return). In Figure 3-3, all those portfolios that have a lower expected return than the minimum risk portfolio are inefficient. The following code derives the efficient portfolios in risk-return space and plots them as seen in Figure 3-4. The set of all efficient portfolios is called the efficient frontier, and agents will only choose a portfolio that lies on the efficient frontier: In [69]: cons = [{'type': 'eq', 'fun': lambda phi: np.sum(phi) - 1}, {'type': 'eq', 'fun': lambda phi: mu_phi(phi) - target}] In [70]: bnds = ((0, 1), (0, 1)) In [71]: targets = np.linspace(mu_phi(min_var['x']), 0.16) In [72]: frontier = [] for target in targets: phi_eff = minimize(sigma_phi, (0.5, 0.5), constraints=cons, bounds=bnds)['x'] frontier.append((sigma_phi(phi_eff), mu_phi(phi_eff))) frontier = np.array(frontier) In [73]: plt.figure(figsize=(10, 6)) plt.plot(frontier[:, 0], frontier[:, 1], 'mo', ms=5, label='efficient frontier') plt.plot(sigma_phi(min_var['x']), mu_phi(min_var['x']), '^', ms=12.5, label='minimum volatility') plt.plot(sigma_phi(max_sharpe['x']), mu_phi(max_sharpe['x']), 'v', ms=12.5, label='maximum Sharpe ratio')

Mean-Variance Portfolio Theory

|

81

plt.xlabel('expected volatility') plt.ylabel('expected return') plt.legend();

The new constraint fixes a target level for the expected return. Generates the set of target expected returns. Derives the minimum volatility portfolio given a target expected return.

Figure 3-4. Efficient frontier

Capital Asset Pricing Model The capital asset pricing model (CAPM) is one of the most widely documented and applied models in finance. At its core, it relates in linear fashion the expected return for a single stock to the expected return of the market portfolio, usually approximated by a broad stock index such as the S&P 500. The model dates back to the pioneering work of Sharpe (1964) and Lintner (1965). Jones (2012, ch. 9) describes the CAPM in relation to MVP as follows: Capital market theory is a positive theory in that it hypothesizes how investors do behave rather than how investors should behave, as in the case of modern portfolio theory (MVP). It is reasonable to view capital market theory as an extension of portfo‐ lio theory, but it is important to understand that MVP is not based on the validity, or lack thereof, of capital market theory. The specific equilibrium model of interest to many investors is known as the capital asset pricing model, typically referred to as the CAPM. It allows us to assess the rele‐ vant risk of an individual security as well as to assess the relationship between risk and

82

|

Chapter 3: Normative Finance

the returns expected from investing. The CAPM is attractive as an equilibrium model because of its simplicity and its implications.

Assumptions and Results Assume the static model economy from the previous section ℳ N = Ω, ℱ , P , � with N traded assets and all simplifying assumptions. In the CAPM, agents are assumed to invest according to MVP, caring only about the risk and return statistics of risky assets over one period. In a capital market equilibrium, all available assets are held by all agents and the mar‐ kets clear. Since agents are assumed to be identical in that they use MVP to form their efficient portfolios, this implies that all agents must hold the same efficient portfolio (in terms of composition) since the set of tradable assets is the same for every agent. In other words, the market portfolio (set of tradable assets) must lie on the efficient frontier. If this were not the case, the market could not be in equilibrium. What is the mechanism to obtain a capital market equilibrium? Today’s prices of the tradable assets are the mechanism to make sure that markets clear. If agents do not demand enough of a tradable asset, its price needs to decrease. If demand is higher than supply, its price needs to increase. If prices are set correctly, demand and supply are equal for every tradable asset. While MVP takes the prices of tradable assets as given, the CAPM is a theory and model about what the equilibrium price of an asset should be, given its risk-return characteristics. The CAPM assumes the existence of (at least) one risk-free asset in which every agent can invest any amount and which earns the risk-free rate of r . Every agent will there‐ fore hold a combination of the market portfolio and the risk-free asset in equilibrium, something known as the two fund separation theorem.9 The set of all such portfolios is called the capital market line (CML). Figure 3-5 shows the CML schematically. Port‐ folios to the right of the market portfolio are only achievable if agents are allowed to sell the risk-free asset short and to borrow money that way: In [74]: plt.figure(figsize=(10, 6)) plt.plot((0, 0.3), (0.01, 0.22), label='capital market line') plt.plot(0, 0.01, 'o', ms=9, label='risk-less asset') plt.plot(0.2, 0.15, '^', ms=9, label='market portfolio') plt.annotate('$(0, \\bar{r})$', (0, 0.01), (-0.01, 0.02)) plt.annotate('$(\sigma_M, \mu_M)$', (0.2, 0.15), (0.19, 0.16)) plt.xlabel('expected volatility') plt.ylabel('expected return') plt.legend();

9 For further details, see Jones (2012, ch. 9).

Capital Asset Pricing Model

|

83

Figure 3-5. Capital market line (CML) If σ M, μM are the expected volatility and return of the market portfolio, the capital market line relating the expected portfolio return μ to the expected volatility σ is defined by the following: μ=r+

μM − r σ σM

The following expression is called the market price of risk: μM − r σM

It expresses how much more expected return in equilibrium is needed for an agent to bear one unit more of risk. The CAPM then relates the expected return of any tradable risky asset n = 1, 2, ..., N to the expected return of the market portfolio as follows: μn = r + βn μM − r

84

|

Chapter 3: Normative Finance

Here, βn is defined by the covariance of the market portfolio with the risky asset n divided by the variance of the market portfolio itself: βn =

σ M, n σ 2M

When βn = 0, the expected return according to the CAPM formula is the risk-free rate. The higher βn is, the higher the expected return for the risky asset will be. βn measures risk that is nondiversifiable. This type of risk is also called market risk or systemic risk. According to the CAPM, this is the only risk for which an agent should be rewarded with a higher expected return.

Numerical Example Assume the static model economy with three possible future states from before ℳ 3 = Ω, ℱ , P , � with the opportunity to borrow and lend at a risk-free rate of r = 0.0025. The two risky assets S, T are available in quantities of 0.8 and 0.2, respectively.

Capital market line Figure 3-6 shows the efficient frontier, the market portfolio, the risk-less asset, and the resulting capital market line in risk-return space: In [75]: phi_M = np.array((0.8, 0.2)) In [76]: mu_M = mu_phi(phi_M) mu_M Out[76]: 0.10666666666666666 In [77]: sigma_M = sigma_phi(phi_M) sigma_M Out[77]: 0.39474323581566567 In [78]: r = 0.0025 In [79]: plt.figure(figsize=(10, 6)) plt.plot(frontier[:, 0], frontier[:, 1], 'm.', ms=5, label='efficient frontier') plt.plot(0, r, 'o', ms=9, label='risk-less asset') plt.plot(sigma_M, mu_M, '^', ms=9, label='market portfolio') plt.plot((0, 0.6), (r, r + ((mu_M - r) / sigma_M) * 0.6), 'r', label='capital market line', lw=2.0) plt.annotate('$(0, \\bar{r})$', (0, r), (-0.015, r + 0.01)) plt.annotate('$(\sigma_M, \mu_M)$', (sigma_M, mu_M), (sigma_M - 0.025, mu_M + 0.01)) plt.xlabel('expected volatility')

Capital Asset Pricing Model

|

85

plt.ylabel('expected return') plt.legend();

Figure 3-6. Capital market line with two risky assets

Optimal portfolio Assume an agent with an expected utility function defined over future payoffs as follows: U:�

ℝ+, x

b P P � u x = � x − x2 2

Here, b > 0. After some transformations, the expected utility function can then be expressed over risk-return combinations: U : ℝ+ × ℝ+

86

|

ℝ, σ, μ

Chapter 3: Normative Finance

μ−

b 2 σ + μ2 2

Specific Quadratic Utility Function Although the MVP theory and the CAPM both assume that invest‐ ors only care about one period portfolio risk and return, this assumption is in general only consistent with the EUT when a spe‐ cific form of the Bernoulli utility function is given: the quadratic utility. This type of Bernoulli function is almost exclusively men‐ tioned and used in the context of MVP theory. Beyond that, its par‐ ticular form and characteristics are usually considered inappropriate. Neither the assumption of normally distributed asset returns nor the quadratic utility function seems to be an “ele‐ gant” way of reconciling the inconsistency between EUT on the one hand and MVP theory and the CAPM on the other hand.

What portfolio combination would the agent choose on the CML? A straightforward utility maximization, implemented in Python, yields the answer. To this end, fix the parameter b = 1: In [80]: def U(p): mu, sigma = p return mu - 1 / 2 * (sigma ** 2 + mu ** 2) In [81]: cons = {'type': 'eq', 'fun': lambda p: p[0] - (r + (mu_M - r) / sigma_M * p[1])} In [82]: opt = minimize(lambda p: -U(p), (0.1, 0.3), constraints=cons) In [83]: opt Out[83]:

fun: jac: message: nfev: nit: njev: status: success: x:

-0.034885186826739426 array([-0.93256102, 0.24608851]) 'Optimization terminated successfully.' 8 2 2 0 True array([0.06743897, 0.2460885 ])

The utility function in risk-return space The condition that the portfolio be on the CML

Indifference curves A visual analysis can illustrate the optimal decision making of the agent. Fixing a util‐ ity level for the agent, one can plot indifference curves in risk-return space. An opti‐ mal portfolio is found when an indifference curve is tangent to the CML. Any other indifference curve (not touching the CML or cutting the CML twice) cannot identify an optimal portfolio. Capital Asset Pricing Model

|

87

First, here is some symbolic Python code that transforms the utility function in riskreturn space into a functional relationship between μ and σ for a fixed utility level v and a fixed parameter value b. Figure 3-7 shows two indifference curves. Every σ, μ combination on such an indifference curve yields the same utility; the agent is indif‐ ferent between such portfolios: In [84]: from sympy import * init_printing(use_unicode=False, use_latex=False) In [85]: mu, sigma, b, v = symbols('mu sigma b v') In [86]: sol = solve('mu - b / 2 * (sigma ** 2 + mu ** 2) - v', mu) In [87]: sol Out[87]:

_________________________ _________________________ / 2 2 / 2 2 1 - \/ - b *sigma - 2*b*v + 1 \/ - b *sigma - 2*b*v + 1 + 1 [--------------------------------, --------------------------------] b b

In [88]: u1 = sol[0].subs({'b': 1, 'v': 0.1}) u1 Out[88]: ______________ / 2 1 - \/ 0.8 - sigma In [89]: u2 = sol[0].subs({'b': 1, 'v': 0.125}) u2 Out[89]: _______________ / 2 1 - \/ 0.75 - sigma In [90]: f1 = lambdify(sigma, u1) f2 = lambdify(sigma, u2) In [91]: sigma_ = np.linspace(0.0, 0.5) u1_ = f1(sigma_) u2_ = f2(sigma_) In [92]: plt.figure(figsize=(10, 6)) plt.plot(sigma_, u1_, label='$v=0.1$') plt.plot(sigma_, u2_, '--', label='$v=0.125$') plt.xlabel('expected volatility') plt.ylabel('expected return') plt.legend();

Defines SymPy symbols Solves the utility function for μ

88

|

Chapter 3: Normative Finance

Substitutes numerical values for b, v Generates callable functions from the resulting equations Specifies values for σ over which to evaluate the functions Evaluates the callable functions for the two different utility levels

Figure 3-7. Indifference curves in risk-return space In a next step, the indifference curves need to be combined with the CML to find out visually what the optimal portfolio choice of the agent is. Making use of the previous numerical optimization results, Figure 3-8 shows the optimal portfolio—the point at which the indifference curve is tangent to the CML. Figure 3-8 shows that the agent indeed chooses a mixture of the market portfolio and the risk-less asset: In [93]: u = sol[0].subs({'b': 1, 'v': -opt['fun']}) u Out[93]: ____________________________ / 2 1 - \/ 0.930229626346521 - sigma In [94]: f = lambdify(sigma, u) In [95]: u_ = f(sigma_) In [96]: plt.figure(figsize=(10, 6)) plt.plot(0, r, 'o', ms=9, label='risk-less asset') plt.plot(sigma_M, mu_M, '^', ms=9, label='market portfolio') plt.plot(opt['x'][1], opt['x'][0], 'v', ms=9, label='optimal portfolio') plt.plot((0, 0.5), (r, r + (mu_M - r) / sigma_M * 0.5),

Capital Asset Pricing Model

|

89

label='capital market line', lw=2.0) plt.plot(sigma_, u_, '--', label='$v={}$'.format(-round(opt['fun'], 3))) plt.xlabel('expected volatility') plt.ylabel('expected return') plt.legend();

Defines the indifference curve for the optimal utility level Derives numerical values to plot the indifference curve

Figure 3-8. Optimal portfolio on the CML The topics presented in this sub-section are usually discussed under capital market theory (CMT). The CAPM is part of that theory and shall be illustrated by the use of real financial time series data in the next chapter.

Arbitrage Pricing Theory Early on, shortcomings of the CAPM were observed and then addressed in the finance literature. One of the major generalizations of the CAPM is the arbitrage pricing theory (APT) as proposed in Ross (1971) and Ross (1976). Ross (1976) intro‐ duces his paper as follows: The purpose of this paper is to examine rigorously the arbitrage model of capital asset pricing developed in Ross (1971). The arbitrage model was proposed as an alternative to the mean variance capital asset pricing model, introduced by Sharpe, Lintner, and Treynor, that has become the major analytic tool for explaining phenomena observed in capital markets for risky assets.

90

|

Chapter 3: Normative Finance

Assumptions and Results The APT is a generalization of the CAPM to multiple risk factors. In that sense, APT does not assume that the market portfolio is the only relevant risk factor; there are rather multiple types of risk that together are assumed to drive the performance (expected returns) of a stock. Such risk factors might include size, volatility, value, and momentum.10 Beyond this major difference, the model relies on similar assump‐ tions, such as that markets are perfect, that (unlimited) borrowing and lending are possible at the same constant rate, and so on. In its original dynamic version, as found in Ross (1976), the APT takes on the following form: yt = a + B f t + �t

Here, yt is the vector of M observed variables—say, the expected returns of M differ‐ ent stocks—at time t: y1t yt =

y2t ⋮ ytM

a is the vector of M constant terms: a1 a=

a2 ⋮ aM

10 See Bender et al. (2013) for more background information about factors used in practice.

Arbitrage Pricing Theory

|

91

f t is the vector of F factors at time t: f 1t ft =

f 2t ⋮ f Ft

B is the M × F matrix of the so-called factor loadings: b11 b12 ... b1F B=

b21 b22 ... b2F ⋮

⋮

bM1 bM2

⋱ ⋮ ... bMF

Finally, �t is the vector of M sufficiently independent residual terms: 1

�t �t =

2

�t ⋮

M

�t

Jones (2012, ch. 9) describes the difference between the CAPM and the APT as follows: Similar to the CAPM, or any other asset pricing model, APT posits a relationship between expected return and risk. It does so, however, using different assumptions and procedures. Very importantly, APT is not critically dependent on an underlying mar‐ ket portfolio as is the CAPM, which predicts that only market risk influences expected returns. Instead, APT recognizes that several types of risk may affect security returns.

Both the CAPM and the APT relate the output variables with the relevant input fac‐ tors in linear fashion. From an econometric point of view, both models are imple‐ mented based on linear ordinary least-squares (OLS) regression. While the CAPM can be implemented based on univariate linear OLS regression, the APT requires multivariate OLS regression.

92

|

Chapter 3: Normative Finance

Numerical Example The following numerical example casts the APT in a static model, although the for‐ mulation given previously is a dynamic one. Assume the static model economy with three possible future states from the previous section, ℳ 3 = Ω, ℱ , P , � . Assume that the two risky assets are now the relevant risk factors in the economy and intro‐ duce a third asset V with the following future payoff: 12 V 1 = 15 7

Although two linearly independent vectors, such as S1, T 1, cannot form a basis of ℝ3, they can nevertheless be used for an OLS regression to approximate the payoff V 1. The following Python code implements the OLS regression: In [97]: M1 Out[97]: array([[20, 1], [10, 12], [ 5, 13]]) In [98]: M0 Out[98]: array([10, 10]) In [99]: V1 = np.array((12, 15, 7)) In [100]: reg = np.linalg.lstsq(M1, V1, rcond=-1)[0] reg Out[100]: array([0.6141665 , 0.50030531]) In [101]: np.dot(M1, reg) Out[101]: array([12.78363525, 12.14532872,

9.57480155])

In [102]: np.dot(M1, reg) - V1 Out[102]: array([ 0.78363525, -2.85467128,

2.57480155])

In [103]: V0 = np.dot(M0, reg) V0 Out[103]: 11.144718094850402

The optimal regression parameters can be interpreted as factor loadings. The two factors are not enough to explain the payoff V 1; the replication is imper‐ fect, and the residual values are nonzero.

Arbitrage Pricing Theory

|

93

The factor loadings can be used to estimate an arbitrage-free price V 0 for the risky asset V . Obviously, the two factors are not enough to fully “explain” the payoff V 1. This is not too surprising given standard results from linear algebra.11 What about adding a third risk factor U to the model economy? Assume that the third risk factor U is defined by U 0 = 10 and the following: 12 U1 = 5 11

Now, the three risk factors together can explain (replicate) the payoff V 1 fully (exactly): In [104]: U0 = 10 U1 = np.array((12, 5, 11)) In [105]: M0_ = np.array((S0, T0, U0)) In [106]: M1_ = np.concatenate((M1.T, np.array([U1,]))).T In [107]: M1_ Out[107]: array([[20, 1, 12], [10, 12, 5], [ 5, 13, 11]]) In [108]: np.linalg.matrix_rank(M1_) Out[108]: 3 In [109]: reg = np.linalg.lstsq(M1_, V1, rcond=-1)[0] reg Out[109]: array([ 0.9575179 , 0.72553699, -0.65632458]) In [110]: np.allclose(np.dot(M1_, reg), V1) Out[110]: True In [111]: V0_ = np.dot(M0_, reg) V0_ Out[111]: 10.267303102625307

Augmented market price vector. Augmented market payoff matrix with full rank. 11 Of course, the payoff V might lie (incidentally) in the span of the two factor payoff vectors S , T .

1

94

|

Chapter 3: Normative Finance

1

1

Exact replication of V 1. Residual values are zero. Unique arbitrage-free price for the risky asset V . The example here resembles the one presented in “Uncertainty and Risk” on page 62 in that enough risk factors (tradable assets) can be used to derive an arbitrage-free price for a traded asset. APT does not necessarily require that perfect replication is possible; its very model formulation contains residual values. However, if perfect rep‐ lication is possible, then the residual terms are zero, as in the previous example with three risk factors.

Conclusions Some of the early theories and models from the 1940s through the 1970s, in particu‐ lar those presented in this chapter, are still central topics of finance textbooks and are still used in financial practice. One reason for this is that many of those mostly normative theories and models have a strong intellectual appeal to students, academ‐ ics, and practitioners alike. They somehow “simply seem to make sense.” Using Python, numerical examples for the models presented are easily created, analyzed, and visualized. Despite theories and models such as MVP and CAPM being intellectually appealing, easy to implement, and mathematically elegant, it is surprising that they are still so popular today, for a few reasons. First, the popular theories and models presented in this chapter have hardly any meaningful empirical support. Second, some of the theo‐ ries and models are even theoretically inconsistent with each other in a number of ways. Third, there has been continuous progress on the theoretical and modeling fronts of finance, such that alternative theories and models are available. Fourth, modern computational and empirical finance can rely on almost unlimited data sour‐ ces and almost unlimited computational power, making concise, parsimonious, and elegant mathematical models and results less and less relevant. The next chapter analyzes some of the theories and models introduced in this chapter on the basis of real financial data. While in the early years of quantitative finance, data was a scarce resource, today even students have access to a wealth of financial data and open source tools that allow the comprehensive analysis of financial theories and models based on real-world data. Empirical finance has always been an impor‐ tant sister discipline to theoretical finance. However, financial theory has usually driven empirical finance to a large extent. The new area of data-driven finance might lead to a lasting shift in the relative importance of theory as compared to data in finance.

Conclusions

|

95

References Books and papers cited in this chapter: Bender, Jennifer et al. 2013. “Foundations of Factor Investing.” MSCI Research Insight. http://bit.ly/aiif_factor_invest. Calvello, Angelo. 2020. “Fund Managers Must Embrace AI Disruption.” Financial Times, January 15, 2020. http://bit.ly/aiif_ai_disrupt. Eichberger, Jürgen, and Ian R. Harper. 1997. Financial Economics. New York: Oxford University Press. Fishburn, Peter. 1968. “Utility Theory.” Management Science 14 (5): 335-378. Fama, Eugene F. and Kenneth R. French. 2004. “The Capital Asset Pricing Model: Theory and Evidence.” Journal of Economic Perspectives 18 (3): 25-46. Halevy, Alon, Peter Norvig, and Fernando Pereira. 2009. “The Unreasonable Effec‐ tiveness of Data.” IEEE Intelligent Systems, Expert Opinion. Hilpisch, Yves. 2015. Derivatives Analytics with Python: Data Analysis, Models, Simu‐ lation, Calibration, and Hedging. Wiley Finance. Jacod, Jean, and Philip Protter. 2004. Probability Essentials. 2nd ed. Berlin: Springer. Johnstone, David and Dennis Lindley. 2013. “Mean-Variance and Expected Utility: The Borch Paradox.” Statistical Science 28 (2): 223-237. Jones, Charles P. 2012. Investments: Analysis and Management. 12th ed. Hoboken: John Wiley & Sons. Karni, Edi. 2014. “Axiomatic Foundations of Expected Utility and Subjective Proba‐ bility.” In Handbook of the Economics of Risk and Uncertainty, edited by Mark J. Machina and W. Kip Viscusi, 1-39. Oxford: North Holland. Lintner, John. 1965. “The Valuation of Risk Assets and the Selection of Risky Invest‐ ments in Stock Portfolios and Capital Budgets.” Review of Economics and Statistics 47 (1): 13-37. Markowitz, Harry. 1952. “Portfolio Selection.” Journal of Finance 7 (1): 77-91. Pratt, John W. 1964. “Risk Aversion in the Small and in the Large.” Econometrica 32 (1/2): 122-136. Ross, Stephen A. 1971. “Portfolio and Capital Market Theory with Arbitrary Prefer‐ ences and Distributions: The General Validity of the Mean-Variance Approach in Large Markets.” Working Paper No. 12-72, Rodney L. White Center for Financial Research. ⸻. 1976. “The Arbitrage Theory of Capital Asset Pricing.” Journal of Economic Theory 13: 341-360.

96

|

Chapter 3: Normative Finance

Rubinstein, Mark. 2006. A History of the Theory of Investments—My Annotated Bib‐ liography. Hoboken: Wiley Finance. Sharpe, William F. 1964. “Capital Asset Prices: A Theory of Market Equilibrium under Conditions of Risk.” Journal of Finance 19 (3): 425-442. ⸻. 1966. “Mutual Fund Performance.” Journal of Business 39 (1): 119-138. Varian, Hal R. 2010. Intermediate Microeconomics: A Modern Approach. 8th ed. New York & London: W.W. Norton & Company. von Neumann, John, and Oskar Morgenstern. 1944. Theory of Games and Economic Behavior. Princeton: Princeton University Press.

References

|

97

CHAPTER 4

Data-Driven Finance

If artificial intelligence is the new electricity, big data is the oil that powers the generators. —Kai-Fu Lee (2018) Nowadays, analysts sift through non-traditional information such as satellite imagery and credit card data, or use artificial intelligence techniques such as machine learning and natural language processing to glean fresh insights from traditional sources such as economic data and earnings-call transcripts. —Robin Wigglesworth (2019)

This chapter discusses central aspects of data-driven finance. For the purposes of this book, data-driven finance is understood to be a financial context (theory, model, application, and so on) that is primarily driven by and based on insights gained from data. “Scientific Method” on page 100 discusses the scientific method, which is about gen‐ erally accepted principles that should guide scientific effort. “Financial Econometrics and Regression” on page 101 is about financial econometrics and related topics. “Data Availability” on page 104 sheds light on which types of (financial) data are available today and in what quality and quantity via programmatic APIs. “Normative Theories Revisited” on page 117 revisits the normative theories of Chapter 3 and ana‐ lyzes them based on real financial time series data. Also based on real financial data, “Debunking Central Assumptions” on page 143 debunks two of the most commonly found assumptions in financial models and theories: normality of returns and linear relationships.

99

Scientific Method The scientific method refers to a set of generally accepted principles that should guide any scientific project. Wikipedia defines the scientific method as follows: The scientific method is an empirical method of acquiring knowledge that has charac‐ terized the development of science since at least the 17th century. It involves careful observation, applying rigorous skepticism about what is observed, given that cognitive assumptions can distort how one interprets the observation. It involves formulating hypotheses, via induction, based on such observations; experimental and measurement-based testing of deductions drawn from the hypotheses; and refinement (or elimination) of the hypotheses based on the experimental findings. These are principles of the scientific method, as distinguished from a definitive series of steps applicable to all scientific enterprises.

Given this definition, normative finance, as discussed in Chapter 3, is in stark contrast to the scientific method. Normative financial theories mostly rely on assumptions and axioms in combination with deduction as the major analytical method to arrive at their central results. • Expected utility theory (EUT) assumes that agents have the same utility function no matter what state of the world unfolds and that they maximize expected utility under conditions of uncertainty. • Mean-variance portfolio (MVP) theory describes how investors should invest under conditions of uncertainty assuming that only the expected return and the expected volatility of a portfolio over one period count. • The capital asset pricing model (CAPM) assumes that only the nondiversifiable market risk explains the expected return and the expected volatility of a stock over one period. • Arbitrage pricing theory (APT) assumes that a number of identifiable risk factors explains the expected return and the expected volatility of a stock over time; admittedly, compared to the other theories, the formulation of APT is rather broad and allows for wide-ranging interpretations. What characterizes the aforementioned normative financial theories is that they were originally derived under certain assumptions and axioms using “pen and paper” only, without any recourse to real-world data or observations. From a historical point of view, many of these theories were rigorously tested against real-world data only long after their publication dates. This can be explained primarily with better data availa‐ bility and increased computational capabilities over time. After all, data and compu‐ tation are the main ingredients for the application of statistical methods in practice. The discipline at the intersection of mathematics, statistics, and finance that applies such methods to financial market data is typically called financial econometrics, the topic of the next section. 100

| Chapter 4: Data-Driven Finance

Financial Econometrics and Regression Adapting the definition provided by Investopedia for econometrics, one can define financial econometrics as follows: [Financial] econometrics is the quantitative application of statistical and mathematical models using [financial] data to develop financial theories or test existing hypotheses in finance and to forecast future trends from historical data. It subjects real-world [financial] data to statistical trials and then compares and contrasts the results against the [financial] theory or theories being tested.

Alexander (2008b) provides a thorough and broad introduction to the field of finan‐ cial econometrics. The second chapter of the book covers single- and multifactor models, such as the CAPM and APT. Alexander (2008b) is part of a series of four books called Market Risk Analysis. The first in the series, Alexander (2008a), covers theoretical background concepts, topics, and methods, such as MVP theory and the CAPM themselves. The book by Campbell (2018) is another comprehensive resource for financial theory and related econometric research. One of the major tools in financial econometrics is regression, in both its univariate and multivariate forms. Regression is also a central tool in statistical learning in gen‐ eral. What is the difference between traditional mathematics and statistical learning? Although there is no general answer to this question (after all, statistics is a sub-field of mathematics), a simple example should emphasize a major difference relevant to the context of this book. First is the standard mathematical way. Assume a mathematical function is given as follows: f :ℝ

ℝ+, x

1 2+ x 2

Given multiple values of xi, i = 1, 2, ..., n, one can derive function values for f by applying the above definition: yi = f xi , i = 1, 2, ..., n

The following Python code illustrates this based on a simple numerical example: In [1]: import numpy as np In [2]: def f(x): return 2 + 1 / 2 * x In [3]: x = np.arange(-4, 5) x

Financial Econometrics and Regression

|

101

Out[3]: array([-4, -3, -2, -1,

0,

1,

2,

3,

4])

In [4]: y = f(x) y Out[4]: array([0. , 0.5, 1. , 1.5, 2. , 2.5, 3. , 3.5, 4. ])

Second is the approach taken in statistical learning. Whereas in the preceding exam‐ ple, the function comes first and then the data is derived, this sequence is reversed in statistical learning. Here, the data is generally given and a functional relationship is to be found. In this context, x is often called the independent variable and y the depen‐ dent variable. Consequently, consider the following data: xi, yi , i = 1, 2, ..., n

The problem is to find, for example, parameters α, β such that: f xi ≡ α + βxi = y i ≈ yi, i = 1, 2, ..., n

Another way of writing this is by including residual values �i, i = 1, 2, ..., n: α + βxi + �i = yi, i = 1, 2, ..., n

In the context of ordinary least-squares (OLS) regression, α, β are chosen to minimize the mean-squared error between the approximated values y i and the real values yi. The minimization problem, then, is as follows: min α, β

1n y − yi n ∑i i

2

In the case of simple OLS regression, as described previously, the optimal solutions are known in closed form and are as follows: Cov x, y Var(x) α = y − βx β=

Here, Cov stands for the covariance, Var values of x, y.

for the variance, and x, y for the mean

Returning to the preceding numerical example, these insights can be used to derive optimal parameters α, β and, in this particular case, to recover the original definition of f x : 102

| Chapter 4: Data-Driven Finance

In [5]: x Out[5]: array([-4, -3, -2, -1,

0,

1,

2,

3,

4])

In [6]: y Out[6]: array([0. , 0.5, 1. , 1.5, 2. , 2.5, 3. , 3.5, 4. ]) In [7]: beta = np.cov(x, y, ddof=0)[0, 1] / x.var() beta Out[7]: 0.49999999999999994 In [8]: alpha = y.mean() - beta * x.mean() alpha Out[8]: 2.0 In [9]: y_ = alpha + beta * x In [10]: np.allclose(y_, y) Out[10]: True

β as derived from the covariance matrix and the variance α as derived from β and the mean values

Estimated values y i, i = 1, 2, ..., n, given α, β Checks whether y i, yi values are numerically equal The preceding example and those in Chapter 1 illustrate that the application of OLS regression to a given data set is in general straightforward. There are more reasons why OLS regression has become one of the central tools in econometrics and finan‐ cial econometrics. Among them are the following: Centuries old The least-squares approach, particularly in combination with regression, has been used for more than 200 years.1 Simplicity The mathematics behind OLS regression is easy to understand and easy to imple‐ ment in programming. Scalability There is basically no limit regarding the data size to which OLS regression can be applied.

1 See, for example, Kopf (2015).

Financial Econometrics and Regression

|

103

Flexibility OLS regression can be applied to a wide range of problems and data sets. Speed OLS regression is fast to evaluate, even on larger data sets. Availability Efficient implementations in Python and many other programming languages are readily available. However, as easy and straightforward as the application of OLS regression might be in general, the method rests on a number of assumptions—most of them related to the residuals—that are not always satisfied in practice. Linearity The model is linear in its parameters, with regard to both the coefficients and the residuals. Independence Independent variables are not perfectly (to a high degree) correlated with each other (no multicollinearity). Zero mean The mean value of the residuals is (close to) zero. No correlation Residuals are not (strongly) correlated with the independent variables. Homoscedasticity The standard deviation of the residuals is (almost) constant. No autocorrelation The residuals are not (strongly) correlated with each other. In practice, it is in general quite simple to test for the validity of the assumptions given a specific data set.

Data Availability Financial econometrics is driven by statistical methods, such as regression, and the availability of financial data. From the 1950s to the 1990s, and even into the early 2000s, theoretical and empirical financial research was mainly driven by relatively small data sets compared to today’s standards, and was mostly comprised of end-of-day (EOD) data. Data availability is something that has changed dramatically over the last decade or so, with more and more types of financial and other data avail‐ able in ever increasing granularity, quantity, and velocity.

104

|

Chapter 4: Data-Driven Finance

Programmatic APIs With regard to data-driven finance, what is important is not only what data is avail‐ able but also how it can be accessed and processed. For quite a while now, finance professionals have relied on data terminals from companies such as Refinitiv (see Eikon Terminal) or Bloomberg (see Bloomberg Terminal), to mention just two of the leading providers. Newspapers, magazines, financial reports, and the like have long been replaced by such terminals as the primary source for financial information. However, the sheer volume and variety of data provided by such terminals cannot be consumed systematically by a single user or even large groups of finance professio‐ nals. Therefore, the major breakthrough in data-driven finance is to be seen in the programmatic availability of data via application programming interfaces (APIs) that allow the usage of computer code to select, retrieve, and process arbitrary data sets. The remainder of this section is devoted to the illustration of such APIs by which even academics and retail investors can retrieve a wealth of different data sets. Before such examples are provided, Table 4-1 offers an overview of categories of data that are in general relevant in a financial context, as well as typical examples. In the table, structured data refers to numerical data types that often come in tabular structures, while unstructured data refers to data in the form of standard text that often has no structure beyond headers or paragraphs, for example. Alternative data refers to data types that are typically not considered financial data. Table 4-1. Relevant types of financial data Time Historical

Structured data Unstructured data Alternative data Prices, fundamentals News, texts Web, social media, satellites

Streaming Prices, volumes

News, filings

Web, social media, satellites, Internet of Things

Structured Historical Data First, structured historical data types will be retrieved programmatically. To this end, the following Python code uses the Eikon Data API.2 To access data via the Eikon Data API, a local application, such as Refinitiv Work‐ space, must be running and the API access must be configured on the Python level: In [11]: import eikon as ek import configparser In [12]: c = configparser.ConfigParser() c.read('../aiif.cfg') ek.set_app_key(c['eikon']['app_id'])

2 This data service is only available via a paid subscription.

Data Availability

|

105

2020-08-04 10:30:18,059 P[14938] [MainThread 4521459136] Error on handshake port 9000 : ReadTimeout(ReadTimeout())

If these requirements are met, historical structured data can be retrieved via a single function call. For example, the following Python code retrieves EOD data for a set of symbols and a specified time interval: In [14]: symbols = ['AAPL.O', 'MSFT.O', 'NFLX.O', 'AMZN.O'] In [15]: data = ek.get_timeseries(symbols, fields='CLOSE', start_date='2019-07-01', end_date='2020-07-01')

In [16]: data.info()

DatetimeIndex: 254 entries, 2019-07-01 to 2020-07-01 Data columns (total 4 columns): # Column Non-Null Count Dtype --- ------ -------------- ----0 AAPL.O 254 non-null float64 1 MSFT.O 254 non-null float64 2 NFLX.O 254 non-null float64 3 AMZN.O 254 non-null float64 dtypes: float64(4) memory usage: 9.9 KB

In [17]: data.tail() Out[17]: CLOSE AAPL.O Date 2020-06-25 364.84 2020-06-26 353.63 2020-06-29 361.78 2020-06-30 364.80 2020-07-01 364.11

MSFT.O

NFLX.O

AMZN.O

200.34 196.33 198.44 203.51 204.70

465.91 443.40 447.24 455.04 485.64

2754.58 2692.87 2680.38 2758.82 2878.70

Defines a list of RICs (symbols) to retrieve data for3 Retrieves EOD Close prices for the list of RICs Shows the meta information for the returned DataFrame object Shows the final rows of the DataFrame object

3 RIC stands for Reuters Instrument Code.

106

|

Chapter 4: Data-Driven Finance

Similarly, one-minute bars with OHLC fields can be retrieved with appropriate adjust‐ ments of the parameters: In [18]: data = ek.get_timeseries('AMZN.O', fields='*', start_date='2020-08-03', end_date='2020-08-04', interval='minute') In [19]: data.info()

DatetimeIndex: 911 entries, 2020-08-03 08:01:00 to 2020-08-04 00:00:00 Data columns (total 6 columns): # Column Non-Null Count Dtype --- ------ -------------- ----0 HIGH 911 non-null float64 1 LOW 911 non-null float64 2 OPEN 911 non-null float64 3 CLOSE 911 non-null float64 4 COUNT 911 non-null float64 5 VOLUME 911 non-null float64 dtypes: float64(6) memory usage: 49.8 KB In [20]: data.head() Out[20]: AMZN.O Date 2020-08-03 08:01:00 2020-08-03 08:02:00 2020-08-03 08:03:00 2020-08-03 08:04:00 2020-08-03 08:05:00

HIGH

LOW

OPEN

3190.00 3183.02 3179.91 3184.00 3184.91

3176.03 3176.03 3177.05 3179.91 3182.91

3176.03 3180.00 3179.91 3179.91 3183.30

CLOSE COUNT 3178.17 3177.01 3177.05 3184.00 3184.00

18.0 15.0 5.0 8.0 12.0

VOLUME 383.0 513.0 14.0 102.0 403.0

Retrieves one-minute bars with all available fields for a single RIC

Data Availability

|

107

One can retrieve more than structured financial time series data from the Eikon Data API. Fundamental data can also be retrieved for a number of RICs and a number of different data fields at the same time, as the following Python code illustrates: In [21]: data_grid, err = ek.get_data(['AAPL.O', 'IBM', 'GOOG.O', 'AMZN.O'], ['TR.TotalReturnYTD', 'TR.WACCBeta', 'YRHIGH', 'YRLOW', 'TR.Ebitda', 'TR.GrossProfit']) In [22]: data_grid Out[22]: Instrument YTD Total Return 0 AAPL.O 49.141271 1 IBM -5.019570 2 GOOG.O 10.278829 3 AMZN.O 68.406897

0 1 2 3

Beta 1.221249 1.208156 1.067084 1.338106

YRHIGH 425.66 158.75 1586.99 3344.29

YRLOW 192.5800 90.5600 1013.5361 1626.0318

EBITDA 7.647700e+10 1.898600e+10 4.757900e+10 3.025600e+10

Gross Profit 98392000000 36488000000 89961000000 114986000000

Retrieves data for multiple RICs and multiple data fields

Programmatic Data Availability Basically all structured financial data is available nowadays in pro‐ grammatic fashion. Financial time series data, in this context, is the paramount example. However, other structured data types such as fundamental data are available in the same way, simplifying the work of quantitative analysts, traders, portfolio managers, and the like significantly.

Structured Streaming Data Many applications in finance require real-time structured data, such as in algorithmic trading or market risk management. The following Python code makes use of the API of the Oanda Trading Platform and streams in real time a number of time stamps, bid quotes, and ask quotes for the Bitcoin price in USD: In [23]: import tpqoa In [24]: oa = tpqoa.tpqoa('../aiif.cfg') In [25]: oa.stream_data('BTC_USD', stop=5) 2020-08-04T08:30:38.621075583Z 11298.8 11334.8 2020-08-04T08:30:50.485678488Z 11298.3 11334.3 2020-08-04T08:30:50.801666847Z 11297.3 11333.3

108

| Chapter 4: Data-Driven Finance

\

2020-08-04T08:30:51.326269990Z 11296.0 11332.0 2020-08-04T08:30:54.423973431Z 11296.6 11332.6

Connects to the Oanda API Streams a fixed number of ticks for a given symbol Printing out the streamed data fields is, of course, only for illustration. Certain finan‐ cial applications might require sophisticated processing of the retrieved data and the generation of signals or statistics, for instance. Particularly during weekdays and trad‐ ing hours, the number of price ticks streamed for financial instruments increases steadily, demanding powerful data processing capabilities on the end of financial institutions that need to process such data in real time or at least in near-real time (“near time”). The significance of this observation becomes clear when looking at Apple Inc. stock prices. One can calculate that there are roughly 252 · 40 = 10, 080 EOD closing quotes for the Apple stock over a period of 40 years. (Apple Inc. went public on December 12, 1980.) The following code retrieves tick data for the Apple stock price for one hour only. The retrieved data set, which might not even be complete for the given time interval, has 50,000 data rows, or five times as many tick quotes as the EOD quotes accumulated over 40 years of trading: In [26]: data = ek.get_timeseries('AAPL.O', fields='*', start_date='2020-08-03 15:00:00', end_date='2020-08-03 16:00:00', interval='tick') In [27]: data.info()

DatetimeIndex: 50000 entries, 2020-08-03 15:26:24.889000 to 2020-08-03 15:59:59.762000 Data columns (total 2 columns): # Column Non-Null Count Dtype --- ------ -------------- ----0 VALUE 49953 non-null float64 1 VOLUME 50000 non-null float64 dtypes: float64(2) memory usage: 1.1 MB In [28]: data.head() Out[28]: AAPL.O Date 2020-08-03 15:26:24.889 2020-08-03 15:26:24.889 2020-08-03 15:26:24.890 2020-08-03 15:26:24.890 2020-08-03 15:26:24.899

VALUE

VOLUME

439.06 439.08 439.08 439.08 439.10

175.0 3.0 100.0 5.0 35.0

Data Availability

|

109

Retrieves tick data for the Apple stock price

EOD Versus Tick Data Most of the financial theories still applied today have their origin in when EOD data was basically the only type of financial data avail‐ able. Today, financial institutions, and even retail traders and investors, are confronted with never-ending streams of real-time data. The example of Apple stock illustrates that for a single stock during one trading hour, there might be four times as many ticks coming in as the amount of EOD data accumulated over a period of 40 years. This not only challenges actors in financial markets, but also puts into question whether existing financial theories can be applied to such an environment at all.

Unstructured Historical Data Many important data sources in finance provide unstructured data only, such as financial news or company filings. Undoubtedly, machines are much better and faster than humans at crunching large amounts of structured, numerical data. However, recent advances in natural language processing (NLP) make machines better and faster at processing financial news too, for example. In 2020, data service providers ingest roughly 1.5 million news articles on a daily basis. It is clear that this vast amount of text-based data cannot be processed properly by human beings. Fortunately, unstructured data is also to a large extent available these days via pro‐ grammatic APIs. The following Python code retrieves a number of news articles from the Eikon Data API related to the company Tesla, Inc. and its production. One article is selected and shown in full: In [29]: news = ek.get_news_headlines('R:TSLA.O PRODUCTION', date_from='2020-06-01', date_to='2020-08-01', count=7 ) In [30]: news Out[30]: 2020-07-29 2020-07-28 2020-07-23 2020-07-23 2020-07-23 2020-07-23 2020-07-22

versionCreated 11:02:31.276 2020-07-29 11:02:31.276000+00:00 00:59:48.000 2020-07-28 00:59:48+00:00 21:20:36.090 2020-07-23 21:20:36.090000+00:00 08:22:17.000 2020-07-23 08:22:17+00:00 07:08:48.000 2020-07-23 07:46:56+00:00 00:55:54.000 2020-07-23 00:55:54+00:00 21:35:42.640 2020-07-22 22:13:26.597000+00:00

2020-07-29 11:02:31.276

110

|

Chapter 4: Data-Driven Finance

\

text \ Tesla Launches Hiring Spree in China as It Pre...

2020-07-28 2020-07-23 2020-07-23 2020-07-23 2020-07-23 2020-07-22

00:59:48.000 21:20:36.090 08:22:17.000 07:08:48.000 00:55:54.000 21:35:42.640

Tesla hiring in Shanghai as production ramps up Tesla speeds up Model 3 production in Shanghai UPDATE 1-'Please mine more nickel,' Musk urges... 'Please mine more nickel,' Musk urges as Tesla... USA-Tesla choisit le Texas pour la production ... TESLA INC - THE REAL LIMITATION ON TESLA GROWT...

2020-07-29 2020-07-28 2020-07-23 2020-07-23 2020-07-23 2020-07-23 2020-07-22

11:02:31.276 00:59:48.000 21:20:36.090 08:22:17.000 07:08:48.000 00:55:54.000 21:35:42.640

storyId urn:newsml:reuters.com:20200729:nCXG3W8s9X:1 urn:newsml:reuters.com:20200728:nL3N2EY3PG:8 urn:newsml:reuters.com:20200723:nNRAcf1v8f:1 urn:newsml:reuters.com:20200723:nL3N2EU1P9:1 urn:newsml:reuters.com:20200723:nL3N2EU0HH:1 urn:newsml:reuters.com:20200723:nL5N2EU03M:1 urn:newsml:reuters.com:20200722:nFWN2ET120:2

2020-07-29 2020-07-28 2020-07-23 2020-07-23 2020-07-23 2020-07-23 2020-07-22

11:02:31.276 00:59:48.000 21:20:36.090 08:22:17.000 07:08:48.000 00:55:54.000 21:35:42.640

\

sourceCode NS:CAIXIN NS:RTRS NS:SOUTHC NS:RTRS NS:RTRS NS:RTRS NS:RTRS

In [31]: storyId = news['storyId'][1] In [32]: from IPython.display import HTML In [33]: HTML(ek.get_news_story(storyId)[:1148]) Out[33]: Jan 06, 2020 Tesla, Inc.TSLA registered record production and deliveries of 104,891 and 112,000 vehicles, respectively, in the fourth quarter of 2019. Notably, the company's Model S/X and Model 3 reported record production and deliveries in the fourth quarter. The Model S/X division recorded production and delivery volume of 17,933 and 19,450 vehicles, respectively. The Model 3 division registered production of 86,958 vehicles, while 92,550 vehicles were delivered. In 2019, Tesla delivered 367,500 vehicles, reflecting an increase of 50%, year over year, and nearly in line with the company's full-year guidance of 360,000 vehicles.

Retrieves metadata for a number of news articles that fall in the parameter range Selects one storyId for which to retrieve the full text Retrieves the full text for the selected article and shows it Data Availability

|

111

Unstructured Streaming Data In the same way that historical unstructured data is retrieved, programmatic APIs can be used to stream unstructured news data, for example, in real time or at least near time. One such API is available for DNA: the Data, News, Analytics platform from Dow Jones. Figure 4-1 shows the screenshot of a web application that streams “Com‐ modity and Financial News” articles and processes these with NLP techniques in real time.

Figure 4-1. News-streaming application based on DNA (Dow Jones) The news-streaming application has the following main features: Full text The full text of each article is available by clicking on the article header. Keyword summary A keyword summary is created and printed on the screen. 112

|

Chapter 4: Data-Driven Finance

Sentiment analysis Sentiment scores are calculated and visualized as colored arrows. Details become visible through a click on the arrows. Word cloud A word cloud summary bitmap is created, shown as a thumbnail and visible after a click on the thumbnail (see Figure 4-2).

Figure 4-2. Word cloud bitmap shown in news-streaming application

Alternative Data Nowadays, financial institutions, and in particular hedge funds, systematically mine a number of alternative data sources to gain an edge in trading and investing. A recent article by Bloomberg lists, among others, the following alternative data sources: • Web-scraped data • Crowd-sourced data • Credit cards and point-of-sales (POS) systems • Social media sentiment • Search trends • Web traffic • Supply chain data • Energy production data • Consumer profiles • Satellite imagery/geospacial data

Data Availability

|

113

• App installs • Ocean vessel tracking • Wearables, drones, Internet of Things (IoT) sensors In the following, the usage of alternative data is illustrated by two examples. The first retrieves and processes Apple Inc. press releases in the form of HTML pages. The following Python code makes use of a set of helper functions as shown in “Python Code” on page 156. In the code, a list of URLs is defined, each representing an HTML page with a press release from Apple Inc. The raw HTML code is then retrieved for each press release. Then the raw code is cleaned up, and an excerpt for one press release is printed: In [34]: import nlp import requests In [35]: sources = [ 'https://nr.apple.com/dE0b1T5G3u', 'https://nr.apple.com/dE4c7T6g1K', 'https://nr.apple.com/dE4q4r8A2A', ]

# iPad Pro # MacBook Air # Mac Mini

In [36]: html = [requests.get(url).text for url in sources] In [37]: data = [nlp.clean_up_text(t) for t in html] In [38]: data[0][536:1001] Out[38]: ' display, powerful a12x bionic chip and face id introducing the new ipad pro with all-screen design and next-generation performance. new york apple today introduced the new ipad pro with all-screen design and next-generation performance, marking the biggest change to ipad ever. the all-new design pushes 11-inch and 12.9-inch liquid retina displays to the edges of ipad pro and integrates face id to securely unlock ipad with just a glance.1 the a12x bionic chip w'

Imports the NLP helper functions Defines the URLs for the three press releases Retrieves the raw HTML codes for the three press releases Cleans up the raw HTML codes (for example, HTML tags are removed) Prints an excerpt from one press release Of course, defining alternative data as broadly as is done in this section implies that there is a limitless amount of data that one can retrieve and process for financial pur‐ poses. At its core, this is the business of search engines such as the one from Google

114

|

Chapter 4: Data-Driven Finance

LLC. In a financial context, it would be of paramount importance to specify exactly what unstructured alternative data sources to tap into. The second example is about the retrieval of data from the social network Twitter, Inc. To this end, Twitter provides API access to tweets on its platform, provided one has set up a Twitter account appropriately. The following Python code connects to the Twitter API and retrieves and prints the five most recent tweets from my home time‐ line and user timeline, respectively: In [39]: from twitter import Twitter, OAuth In [40]: t = Twitter(auth=OAuth(c['twitter']['access_token'], c['twitter']['access_secret_token'], c['twitter']['api_key'], c['twitter']['api_secret_key']), retry=True) In [41]: l = t.statuses.home_timeline(count=5) In [42]: for e in l: print(e['text']) The Bank of England is effectively subsidizing polluting industries in its pandemic rescue program, a think tank sa… https://t.co/Fq5jl2CIcp Cool shared task: mining scientific contributions (by @SeeTedTalk @SoerenAuer and Jennifer D'Souza) https://t.co/dm56DMUrWm Twelve people were hospitalized in Wyoming on Monday after a hot air balloon crash, officials said. Three hot air… https://t.co/EaNBBRXVar President Trump directed controversial Pentagon pick into new role with similar duties after nomination failed https://t.co/ZyXpPcJkcQ Company announcement: Revolut launches Open Banking for its 400,000 Italian... https://t.co/OfvbgwbeJW #fintech In [43]: l = t.statuses.user_timeline(screen_name='dyjh', count=5) In [44]: for e in l: print(e['text']) #Python for #AlgoTrading (focus on the process) & #AI in #Finance (focus on prediction methods) will complement eac… https://t.co/P1s8fXCp42 Currently putting finishing touches on #AI in #Finance (@OReillyMedia). Book going into production shortly. https://t.co/JsOSA3sfBL Chinatown Is Coming Back, One Noodle at a Time https://t.co/In5kXNeVc5 Alt data industry balloons as hedge funds strive for Covid edge via @FT | "We remain of the view that alternative d… https://t.co/9HtUOjoEdz @Wolf_Of_BTC Just follow me on Twitter (or LinkedIn). Then you will notice for sure when it is out.

Data Availability

|

115

Connects to the Twitter API Retrieves and prints five (most recent) tweets from home timeline Retrieves and prints five (most recent) tweets from user timeline The Twitter API allows also for searches, based on which most recent tweets can be retrieved and processed: In [45]: d = t.search.tweets(q='#Python', count=7) In [46]: for e in d['statuses']: print(e['text']) RT @KirkDBorne: #AI is Reshaping Programming — Tips on How to Stay on Top: https://t.co/CFNu1i352C —— Courses: 1: #MachineLearning — Jupyte… RT @reuvenmlerner: Today, a #Python student's code didn't print: x = 5 if x == 5: print: ('yes!') There was a typo, namely : after pr… RT @GavLaaaaaaaa: Javascript Does Not Need a StringBuilder https://t.co/aS7NzHLO65 #programming #softwareengineering #bigdata #datascience… RT @CodeFlawCo: It is necessary to publish regular updates on Twitter #programmer #coder #developer #technology RT @pak_aims: Learning to C… RT @GavLaaaaaaaa: Javascript Does Not Need a StringBuilder https://t.co/aS7NzHLO65 #programming #softwareengineering #bigdata #datascience…

Searches for tweets with hashtag “Python” and prints the five most recent ones One can also collect a larger number of tweets from a Twitter user and create a sum‐ mary in the form of a word cloud (see Figure 4-3). The following Python code again makes use of the NLP helper functions as shown in “Python Code” on page 156: In [47]: l = t.statuses.user_timeline(screen_name='elonmusk', count=50) In [48]: tl = [e['text'] for e in l] In [49]: tl[:5] Out[49]: ['@flcnhvy @Lindw0rm @cleantechnica True', '@Lindw0rm @cleantechnica Highly likely down the road', '@cleantechnica True fact', '@NASASpaceflight Scrubbed for the day. A Raptor turbopump spin start valve didn’t open, triggering an automatic abo… https://t.co/QDdlNXFgJg',

116

| Chapter 4: Data-Driven Finance

'@Erdayastronaut I’m in the Boca control room. Hop attempt in ~33 minutes.'] In [50]: wc = nlp.generate_word_cloud(' '.join(tl), 35, name='../../images/ch04/musk_twitter_wc.png' )

Retrieves the 50 most recent tweets for the user elonmusk Collects the texts in a list object Shows excerpts for the final five tweets Generates a word cloud summary and shows it

Figure 4-3. Word cloud as summary for larger number of tweets Once a financial practitioner defines the “relevant financial data” to go beyond struc‐ tured financial time series data, the data sources seem limitless in terms of volume, variety, and velocity. The way the tweets are retrieved from the Twitter API is almost in near time since the most recent tweets are accessed in the examples. These and similar API-based data sources therefore provide a never-ending stream of alternative data for which, as previously pointed out, it is important to specify exactly what one is looking for. Otherwise, any financial data science effort might easily drown in too much data and/or too noisy data.

Normative Theories Revisited Chapter 3 introduces normative financial theories such as the MVP theory or the CAPM. For quite a long time, students and academics learning and studying such theories were more or less constrained to the theory itself. With all the available financial data, as discussed and illustrated in the previous section, in combination with powerful open source software for data analysis—such as Python, NumPy, pan das, and so on—it has become pretty easy and straightforward to put financial theo‐ ries to real-world tests. It does not require small teams and larger studies anymore to do so. A typical notebook, internet access, and a standard Python environment suf‐ fice. This is what this section is about. However, before diving into data-driven Normative Theories Revisited

|

117

finance, the following sub-section discusses briefly some famous paradoxes in the context of EUT and how corporations model and predict the behavior of individuals in practice.

Expected Utility and Reality In economics, risk describes a situation in which possible future states and probabili‐ ties for those states to unfold are known in advance to the decision maker. This is the standard assumption in finance and the context of EUT. On the other hand, ambigu‐ ity describes situations in economics in which probabilities, or even possible future states, are not known in advance to a decision maker. Uncertainty subsumes the two different decision-making situations. There is a long tradition of analyzing the concrete decision-making behavior of indi‐ viduals (“agents”) under uncertainty. Innumerable studies and experiments have been conducted to observe and analyze how agents behave when faced with uncertainty as compared to what theories such as EUT predict. For centuries, paradoxa have played an important role in decision-making theory and research. One such paradox, the St. Petersburg paradox, gave rise to the invention of utility functions and EUT in the first place. Daniel Bernoulli presented the paradox—and a solution to it—in 1738. The paradox is based on the following coin tossing game G. An agent is faced with a game during which a (perfect) coin is tossed potentially infinitely many times. If after the first toss heads prevails, the agent receives a payoff of 1 (currency unit). As long as heads is observed, the coin is tossed again. Otherwise the game ends. If heads prevails a second time, the agent receives an additional payoff of 2. If it does a third time, the additional payoff is 4. For the fourth time it is 8, and so on. This is a situation of risk since all possible future states, as well as their associ‐ ated probabilities, are known in advance. The expected payoff of this game is infinite. This can be seen from the following infinite sum of which every element is strictly positive: �G =

1 1 1 1 ·1+ ·2+ ·4+ · 8 + ... = 2 4 8 16

∞

∞

∑ 1 2k − 1 = k ∑= 1 12 = ∞ k=1 k 2

However, faced with such a game, a decision maker in general would be willing to pay a finite sum only to play the game. A major reason for this is the fact that relatively large payoffs only happen with a relatively small probability. Consider the potential payoff W = 511: W = 1 + 2 + 4 + 8 + 16 + 32 + 64 + 128 + 256 = 511

118

| Chapter 4: Data-Driven Finance

The probability of winning such a payoff is pretty low. To be exact, it is only 1 P x = W = 512 = 0.001953125. The probability for such a payoff or a smaller one, on the other hand, is pretty high: Px≤W =

9

1 = 0 . 998046875 ∑ k=1 k 2

In other words, in 998 out of 1,000 games the payoff is 511 or smaller. Therefore, an agent would probably not wager much more than 511 to play this game. The way out of this paradox is the introduction of a utility function with positive but decreasing marginal utility. In the context of the St. Petersburg paradox, this means that there is a function u: ℝ+ ℝ that assigns to every positive payoff x a real value u x . Positive but decreasing marginal utility then formally translates into the following: ∂u > 0 ∂x ∂2u < 0 ∂x2

As seen in Chapter 3, one such candidate function is u x = ln x with: ∂u = ∂x

1 x

∂2u 1 = − 2 2 ∂x x

The expected utility then is finite, as the calculation of the following infinite sum illustrates: �uG =

∞

∑ 1 u 2k − 1 k=1 k 2

=

∞

∑ k=1

ln 2k − 1 2

k

=

∞

∑ k=1

k−1 2k

· ln 2 = ln 2 < ∞

The expected utility of ln 2 = 0.693147 is obviously a pretty small number in com‐ parison to the expected payoff of infinity. Bernoulli utility functions and EUT resolve the St. Petersburg paradox. Other paradoxa, such as the Allais paradox published in Allais (1953), address the EUT itself. This paradox is based on an experiment with four different games that test subjects should rank. Table 4-2 shows the four games A, B, A′, B′ . The ranking is to be done for the two pairs A, B and A′, B′ . The independence axiom postulates that Normative Theories Revisited

|

119

the first row in the table should not have any influence on the ordering of A′, B′ since the payoff is the same for both games. Table 4-2. Games in Allais paradox Probability Game A Game B Game A’ Game B’ 0.66 2,400 2,400 0 0 0.33

2,500

2,400

2,500

2,400

0.01

0

2,400

0

2,400

In experiments, the majority of decision makers rank the games as follows: B ≻ A and A′ ≻ B′. The ranking B ≻ A leads to the following inequalities, where u1 ≡ u 2400 , u2 ≡ u 2500 , u3 ≡ u 0 : u1

> 0 . 66 · u1 + 0 . 33 · u2 + 0 . 01 · u3

0 . 34 · u1 >

0 . 33 · u2 + 0 . 01 · u3

The ranking A′ ≻ B′ in turn leads to the following inequalities: 0 . 33 · u2 + 0 . 01 · u3 > 0 . 33 · u1 + 0 . 01 · u1 0 . 34 · u1

< 0 . 33 · u2 + 0 . 01 · u3

These inequalities obviously contradict each other and lead to the Allais paradox. One possible explanation is that decision makers in general value certainty higher than the typical models, such as EUT, predict. Most people would probably rather choose to receive $1 million with certainty than play a game in which they can win $100 million with a probability of 5%, although there are a number of suitable utility functions available that under EUT would have the decision maker choose the game instead of the certain amount. Another explanation lies in framing decisions and the psychology of decision makers. It is well known that more people would accept a surgery if it has a “95% chance of success” than a “5% chance of death.” Simply changing the wording might lead to behavior that is inconsistent with decision-making theories such as EUT. Another famous paradox addressing shortcomings of EUT in its subjective form, according to Savage (1954, 1972), is the Ellsberg paradox, which dates back to the seminal paper by Ellsberg (1961). It addresses the importance of ambiguity in many real-world decision situations. A standard setting for this paradox comprises two dif‐ ferent urns, both of which contain exactly 100 balls. For urn 1, it is known that it con‐ tains exactly 50 black and 50 red balls. For urn 2, it is only known that it contains black and red balls but not in which proportion. 120

|

Chapter 4: Data-Driven Finance

Test subjects can choose among the following game options: • Game 1: red 1, black 1, or indifferent • Game 2: red 2, black 2, or indifferent • Game 3: red 1, red 2, or indifferent • Game 4: black 1, black 2, or indifferent Here, “red 1,” for example, means that a red ball is drawn from urn 1. Typically, a test subject would answer as follows: • Game 1: indifferent • Game 2: indifferent • Game 3: red 1 • Game 4: black 1 This set of decisions—which is not the only one to be observed but is a common one —exemplifies what is called ambiguity aversion. Since the probabilities for black and red balls, respectively, are not known for urn 2, decision makers prefer a situation of risk instead of ambiguity. The two paradoxa of Allais and Ellsberg show that real test subjects quite often behave contrary to what well-established decision theories in economics predict. In other words, human beings as decision makers can in general not be compared to machines that carefully collect data and then crunch the numbers to make a decision under uncertainty, be it in the form of risk or ambiguity. Human behavior is more complex than most, if not all, theories currently suggest. How difficult and complex it can be to explain human behavior is clear after reading, for example, the 800-page book Behave by Sapolsky (2018). It covers multiple facets of this topic, ranging from biochemical processes to genetics, human evolution, tribes, language, religion, and more, in an integrative manner. If standard economic decision paradigms such as EUT do not explain real-world decision making too well, what alternatives are available? Economic experiments that build the basis for the Allais and Ellsberg paradoxa are a good starting point in learn‐ ing how decision makers behave in specific, controlled situations. Such experiments and their sometimes surprising and paradoxical results have indeed motivated a great number of researchers to come up with alternative theories and models that resolve the paradoxa. The book The Experiment in the History of Economics by Fontaine and Leonard (2005) is about the historical role of experiments in economics. There is, for example, a whole string of literature that addresses issues arising from the Ellsberg paradox. This literature deals with, among other topics, nonadditive probabilities, Choquet integrals, and decision heuristics such as maximizing the minimum payoff

Normative Theories Revisited

|

121

(“max-min”) or minimizing the maximum loss (“min-max”). These alternative approaches have proven superior to EUT, at least in certain decision-making scenar‐ ios. But they are far from being mainstream in finance. What, after all, has proven to be useful in practice? Not too surprisingly, the answer lies in data and machine learning algorithms. The internet, with its billions of users, generates a treasure trove of data describing real-world human behavior, or what is sometimes called revealed preferences. The big data generated on the web has a scale that is multiple orders of magnitude larger than what single experiments can gener‐ ate. Companies such as Amazon, Facebook, Google, and Twitter are able to make bil‐ lions of dollars by recording user behavior (that is, their revealed preferences) and capitalizing on the insights generated by ML algorithms trained on this data. The default ML approach taken in this context is supervised learning. The algorithms themselves are in general theory- and model-free; variants of neural networks are often applied. Therefore, when companies today predict the behavior of their users or customers, more often than not a model-free ML algorithm is deployed. Traditional decision theories like EUT or one of its successors generally do not play a role at all. This makes it somewhat surprising that such theories still, at the beginning of the 2020s, are a cornerstone of most economic and financial theories applied in practice. And this is not even to mention the large number of financial textbooks that cover traditional decision theories in detail. If one of the most fundamental building blocks of financial theory seems to lack meaningful empirical support or practical benefits, what about the financial models that build on top of it? More on this appears in sub‐ sequent sections and chapters.

Data-Driven Predictions of Behavior Standard economic decision theories are intellectually appealing to many, even to those who, faced with a concrete decision under uncertainty, would behave in contrast to the theories’ predictions. On the other hand, big data and model-free, supervised learning approaches prove useful and successful in practice for predicting user and customer behavior. In a financial context, this might imply that one should not really worry about why and how finan‐ cial agents decide the way they decide. One should rather focus on their indirectly revealed preferences based on features data (new information) that describes the state of a financial market and labels data (outcomes) that reflects the impact of the decisions made by financial agents. This leads to a data-driven instead of a theory- or model-driven view of decision making in financial mar‐ kets. Financial agents become data-processing organisms that can be much better modeled, for example, by complex neural networks than, say, a simple utility function in combination with an assumed probability distribution.

122

|

Chapter 4: Data-Driven Finance

Mean-Variance Portfolio Theory Assume a data-driven investor wants to apply MVP theory to invest in a portfolio of technology stocks and wants to add a gold-related exchange-traded fund (ETF) for diversification. Probably, the investor would access relevant historical price data via an API to a trading platform or a data provider. To make the following analysis repro‐ ducible, it relies on a CSV data file stored in a remote location. The following Python code retrieves the data file, selects a number of symbols given the investor’s goal, and calculates log returns from the price time series data. Figure 4-4 compares the nor‐ malized price time series for the selected symbols: In [51]: import numpy as np import pandas as pd from pylab import plt, mpl from scipy.optimize import minimize plt.style.use('seaborn') mpl.rcParams['savefig.dpi'] = 300 mpl.rcParams['font.family'] = 'serif' np.set_printoptions(precision=5, suppress=True, formatter={'float': lambda x: f'{x:6.3f}'}) In [52]: url = 'http://hilpisch.com/aiif_eikon_eod_data.csv' In [53]: raw = pd.read_csv(url, index_col=0, parse_dates=True).dropna() In [54]: raw.info()

DatetimeIndex: 2516 entries, 2010-01-04 to 2019-12-31 Data columns (total 12 columns): # Column Non-Null Count Dtype --- ------ -------------- ----0 AAPL.O 2516 non-null float64 1 MSFT.O 2516 non-null float64 2 INTC.O 2516 non-null float64 3 AMZN.O 2516 non-null float64 4 GS.N 2516 non-null float64 5 SPY 2516 non-null float64 6 .SPX 2516 non-null float64 7 .VIX 2516 non-null float64 8 EUR= 2516 non-null float64 9 XAU= 2516 non-null float64 10 GDX 2516 non-null float64 11 GLD 2516 non-null float64 dtypes: float64(12) memory usage: 255.5 KB In [55]: symbols = ['AAPL.O', 'MSFT.O', 'INTC.O', 'AMZN.O', 'GLD'] In [56]: rets = np.log(raw[symbols] / raw[symbols].shift(1)).dropna() In [57]: (raw[symbols] / raw[symbols].iloc[0]).plot(figsize=(10, 6));

Normative Theories Revisited

|

123

Retrieves historical EOD data from a remote location Specifies the symbols (RICs) to be invested in Calculates the log returns for all time series Plots the normalized financial time series for the selected symbols

Figure 4-4. Normalized financial time series data The data-driven investor wants to first set a baseline for performance as given by an equally weighted portfolio over the whole period of the available data. To this end, the following Python code defines functions to calculate the portfolio return, the portfo‐ lio volatility, and the portfolio Sharpe ratio given a set of weights for the selected sym‐ bols: In [58]: weights = len(rets.columns) * [1 / len(rets.columns)] In [59]: def port_return(rets, weights): return np.dot(rets.mean(), weights) * 252 In [60]: port_return(rets, weights) Out[60]: 0.15694764653018106 In [61]: def port_volatility(rets, weights): return np.dot(weights, np.dot(rets.cov() * 252 , weights)) ** 0.5 In [62]: port_volatility(rets, weights) Out[62]: 0.16106507848480675 In [63]: def port_sharpe(rets, weights): return port_return(rets, weights) / port_volatility(rets, weights)

124

|

Chapter 4: Data-Driven Finance

In [64]: port_sharpe(rets, weights) Out[64]: 0.97443622172255

Equally weighted portfolio Portfolio return Portfolio volatility Portfolio Sharpe ratio (with zero short rate) The investor also wants to analyze which combinations of portfolio risk and return— and consequently Sharpe ratio—are roughly possible by applying Monte Carlo simu‐ lation to randomize the portfolio weights. Short sales are excluded, and the portfolio weights are assumed to add up to 100%. The following Python code implements the simulation and visualizes the results (see Figure 4-5): In [65]: w = np.random.random((1000, len(symbols))) w = (w.T / w.sum(axis=1)).T In [66]: w[:5] Out[66]: array([[ [ [ [ [

0.184, 0.207, 0.313, 0.238, 0.246,

0.157, 0.282, 0.284, 0.181, 0.256,

0.227, 0.258, 0.051, 0.145, 0.315,

0.353, 0.023, 0.340, 0.191, 0.181,

0.079], 0.230], 0.012], 0.245], 0.002]])

In [67]: pvr = [(port_volatility(rets[symbols], weights), port_return(rets[symbols], weights)) for weights in w] pvr = np.array(pvr) In [68]: psr = pvr[:, 1] / pvr[:, 0] In [69]: plt.figure(figsize=(10, 6)) fig = plt.scatter(pvr[:, 0], pvr[:, 1], c=psr, cmap='coolwarm') cb = plt.colorbar(fig) cb.set_label('Sharpe ratio') plt.xlabel('expected volatility') plt.ylabel('expected return') plt.title(' | '.join(symbols));

Simulates portfolio weights adding up to 100% Derives the resulting portfolio volatilities and returns Calculates the resulting Sharpe ratios

Normative Theories Revisited

|

125

Figure 4-5. Simulated portfolio volatilities, returns, and Sharpe ratios The data-driven investor now wants to backtest the performance of a portfolio that was set up at the beginning of 2011. The optimal portfolio composition was derived from the financial time series data available from 2010. At the beginning of 2012, the portfolio composition was adjusted given the available data from 2011, and so on. To this end, the following Python code derives the portfolio weights for every relevant year that maximizes the Sharpe ratio: In [70]: bnds = len(symbols) * [(0, 1),] bnds Out[70]: [(0, 1), (0, 1), (0, 1), (0, 1), (0, 1)] In [71]: cons = {'type': 'eq', 'fun': lambda weights: weights.sum() - 1} In [72]: opt_weights = {} for year in range(2010, 2019): rets_ = rets[symbols].loc[f'{year}-01-01':f'{year}-12-31'] ow = minimize(lambda weights: -port_sharpe(rets_, weights), len(symbols) * [1 / len(symbols)], bounds=bnds, constraints=cons)['x'] opt_weights[year] = ow In [73]: opt_weights Out[73]: {2010: array([ 2011: array([ 2012: array([ 2013: array([ 2014: array([ 2015: array([

126

|

0.366, 0.543, 0.324, 0.012, 0.452, 0.000,

Chapter 4: Data-Driven Finance

0.000, 0.000, 0.056, 0.578]), 0.000, 0.077, 0.000, 0.380]), 0.000, 0.000, 0.471, 0.205]), 0.305, 0.219, 0.464, 0.000]), 0.115, 0.419, 0.000, 0.015]), 0.000, 0.000, 1.000, 0.000]),

2016: array([ 0.150, 0.260, 0.000, 0.058, 0.533]), 2017: array([ 0.231, 0.203, 0.031, 0.109, 0.426]), 2018: array([ 0.000, 0.295, 0.000, 0.705, 0.000])}

Specifies the bounds for the single asset weights Specifies that all weights need to add up to 100% Selects the relevant data set for the given year Derives the portfolio weights that maximize the Sharpe ratio Stores these weights in a dict object The optimal portfolio compositions as derived for the relevant years illustrate that MVP theory in its original form quite often leads to (relative) extreme situations in the sense that one or more assets are not included at all or that even a single asset makes up 100% of the portfolio. Of course, this can be actively avoided by setting, for example, a minimum weight for every asset considered. The results also indicate that this approach leads to significant rebalancings in the portfolio, driven by the previous year’s realized statistics and correlations. To complete the backtest, the following code compares the expected portfolio statis‐ tics (from the optimal composition of the previous year applied to the previous year’s data) with the realized portfolio statistics for the current year (from the optimal com‐ position from the previous year applied to the current year’s data): In [74]: res = pd.DataFrame() for year in range(2010, 2019): rets_ = rets[symbols].loc[f'{year}-01-01':f'{year}-12-31'] epv = port_volatility(rets_, opt_weights[year]) epr = port_return(rets_, opt_weights[year]) esr = epr / epv rets_ = rets[symbols].loc[f'{year + 1}-01-01':f'{year + 1}-12-31'] rpv = port_volatility(rets_, opt_weights[year]) rpr = port_return(rets_, opt_weights[year]) rsr = rpr / rpv res = res.append(pd.DataFrame({'epv': epv, 'epr': epr, 'esr': esr, 'rpv': rpv, 'rpr': rpr, 'rsr': rsr}, index=[year + 1])) In [75]: res Out[75]: 2011 2012 2013 2014 2015 2016

epv 0.157440 0.173279 0.202460 0.181544 0.160340 0.326730

epr 0.303003 0.169321 0.278459 0.368961 0.309486 0.778330

esr 1.924564 0.977156 1.375378 2.032353 1.930190 2.382179

rpv rpr rsr 0.160622 0.133836 0.833235 0.182292 0.161375 0.885256 0.168714 0.166897 0.989228 0.197798 0.026830 0.135645 0.211368 -0.024560 -0.116194 0.296565 0.103870 0.350242

Normative Theories Revisited

|

127

2017 2018 2019

0.106148 0.086548 0.323796

0.090933 0.260702 0.228008

0.856663 3.012226 0.704174

0.079521 0.157337 0.207672

0.230630 0.038234 0.275819

2.900235 0.243004 1.328147

In [76]: res.mean() Out[76]: epv 0.190920 epr 0.309689 esr 1.688320 rpv 0.184654 rpr 0.123659 rsr 0.838755 dtype: float64

Expected portfolio statistics Realized portfolio statistics Figure 4-6 compares the expected and realized portfolio volatilities for the single years. MVP theory does quite a good job in predicting the portfolio volatility. This is also supported by a relatively high correlation between the two time series: In [77]: res[['epv', 'rpv']].corr() Out[77]: epv rpv epv 1.000000 0.765733 rpv 0.765733 1.000000 In [78]: res[['epv', 'rpv']].plot(kind='bar', figsize=(10, 6), title='Expected vs. Realized Portfolio Volatility');

Figure 4-6. Expected versus realized portfolio volatilities

128

|

Chapter 4: Data-Driven Finance

However, the conclusions are the opposite when comparing the expected with the realized portfolio returns (see Figure 4-7). MVP theory obviously fails in predicting the portfolio returns, as is confirmed by the negative correlation between the two time series: In [79]: res[['epr', 'rpr']].corr() Out[79]: epr rpr epr 1.000000 -0.350437 rpr -0.350437 1.000000 In [80]: res[['epr', 'rpr']].plot(kind='bar', figsize=(10, 6), title='Expected vs. Realized Portfolio Return');

Figure 4-7. Expected versus realized portfolio returns Similar, or even worse, conclusions need to be drawn with regard to the Sharpe ratio (see Figure 4-8). For the data-driven investor who aims at maximizing the Sharpe ratio of the portfolio, the theory’s predictions are generally significantly off from the realized values. The correlation between the two time series is even lower than for the returns: In [81]: res[['esr', 'rsr']].corr() Out[81]: esr rsr esr 1.000000 -0.698607 rsr -0.698607 1.000000 In [82]: res[['esr', 'rsr']].plot(kind='bar', figsize=(10, 6), title='Expected vs. Realized Sharpe Ratio');

Normative Theories Revisited

|

129

Figure 4-8. Expected versus realized portfolio Sharpe ratios

Predictive Power of MVP Theory MVP theory applied to real-world data reveals its practical short‐ comings. Without additional constraints, optimal portfolio compo‐ sitions and rebalancings can be extreme. The predictive power with regard to portfolio return and Sharpe ratio is pretty bad in the numerical example, whereas the predictive power with regard to portfolio risk seems acceptable. However, investors generally are interested in risk-adjusted performance measures, such as the Sharpe ratio, and this is the statistic for which MVP theory fails worst in the example.

Capital Asset Pricing Model A similar approach can be applied to put the CAPM to a real-world test. Assume that the data-driven technology investor from before wants to apply the CAPM to derive expected returns for the four technology stocks from before. The following Python code first derives the beta for every stock for a given year, and then calculates the expected return for the stock in the next year, given its beta and the performance of the market portfolio. The market portfolio is approximated by the S&P 500 stock index: In [83]: r = 0.005 In [84]: market = '.SPX' In [85]: rets = np.log(raw / raw.shift(1)).dropna()

130

|

Chapter 4: Data-Driven Finance

In [86]: res = pd.DataFrame() In [87]: for sym in rets.columns[:4]: print('\n' + sym) print(54 * '=') for year in range(2010, 2019): rets_ = rets.loc[f'{year}-01-01':f'{year}-12-31'] muM = rets_[market].mean() * 252 cov = rets_.cov().loc[sym, market] var = rets_[market].var() beta = cov / var rets_ = rets.loc[f'{year + 1}-01-01':f'{year + 1}-12-31'] muM = rets_[market].mean() * 252 mu_capm = r + beta * (muM - r) mu_real = rets_[sym].mean() * 252 res = res.append(pd.DataFrame({'symbol': sym, 'mu_capm': mu_capm, 'mu_real': mu_real}, index=[year + 1]), sort=True) print('{} | beta: {:.3f} | mu_capm: {:6.3f} | mu_real: {:6.3f}' .format(year + 1, beta, mu_capm, mu_real))

Specifies the risk-less short rate Defines the market portfolio Derives the beta of the stock Calculates the expected return given previous year’s beta and current year market portfolio performance Calculates the realized performance of the stock for the current year Collects and prints all results The preceding code provides the following output: AAPL.O ====================================================== 2011 | beta: 1.052 | mu_capm: -0.000 | mu_real: 0.228 2012 | beta: 0.764 | mu_capm: 0.098 | mu_real: 0.275 2013 | beta: 1.266 | mu_capm: 0.327 | mu_real: 0.053 2014 | beta: 0.630 | mu_capm: 0.070 | mu_real: 0.320 2015 | beta: 0.833 | mu_capm: -0.005 | mu_real: -0.047 2016 | beta: 1.144 | mu_capm: 0.103 | mu_real: 0.096 2017 | beta: 1.009 | mu_capm: 0.180 | mu_real: 0.381 2018 | beta: 1.379 | mu_capm: -0.091 | mu_real: -0.071 2019 | beta: 1.252 | mu_capm: 0.316 | mu_real: 0.621

Normative Theories Revisited

|

131

MSFT.O ====================================================== 2011 | beta: 0.890 | mu_capm: 0.001 | mu_real: -0.072 2012 | beta: 0.816 | mu_capm: 0.104 | mu_real: 0.029 2013 | beta: 1.109 | mu_capm: 0.287 | mu_real: 0.337 2014 | beta: 0.876 | mu_capm: 0.095 | mu_real: 0.216 2015 | beta: 0.955 | mu_capm: -0.007 | mu_real: 0.178 2016 | beta: 1.249 | mu_capm: 0.113 | mu_real: 0.113 2017 | beta: 1.224 | mu_capm: 0.217 | mu_real: 0.321 2018 | beta: 1.303 | mu_capm: -0.086 | mu_real: 0.172 2019 | beta: 1.442 | mu_capm: 0.364 | mu_real: 0.440 INTC.O ====================================================== 2011 | beta: 1.081 | mu_capm: -0.000 | mu_real: 0.142 2012 | beta: 0.842 | mu_capm: 0.108 | mu_real: -0.163 2013 | beta: 1.081 | mu_capm: 0.280 | mu_real: 0.230 2014 | beta: 0.883 | mu_capm: 0.096 | mu_real: 0.335 2015 | beta: 1.055 | mu_capm: -0.008 | mu_real: -0.052 2016 | beta: 1.009 | mu_capm: 0.092 | mu_real: 0.051 2017 | beta: 1.261 | mu_capm: 0.223 | mu_real: 0.242 2018 | beta: 1.163 | mu_capm: -0.076 | mu_real: 0.017 2019 | beta: 1.376 | mu_capm: 0.347 | mu_real: 0.243 AMZN.O ====================================================== 2011 | beta: 1.102 | mu_capm: -0.001 | mu_real: -0.039 2012 | beta: 0.958 | mu_capm: 0.122 | mu_real: 0.374 2013 | beta: 1.116 | mu_capm: 0.289 | mu_real: 0.464 2014 | beta: 1.262 | mu_capm: 0.135 | mu_real: -0.251 2015 | beta: 1.473 | mu_capm: -0.013 | mu_real: 0.778 2016 | beta: 1.122 | mu_capm: 0.102 | mu_real: 0.104 2017 | beta: 1.118 | mu_capm: 0.199 | mu_real: 0.446 2018 | beta: 1.300 | mu_capm: -0.086 | mu_real: 0.251 2019 | beta: 1.619 | mu_capm: 0.408 | mu_real: 0.207

Figure 4-9 compares the predicted (expected) return for a single stock, given the beta from the previous year and market portfolio performance of the current year, with the realized return of the stock for the current year. Obviously, the CAPM in its origi‐ nal form does not prove really useful in predicting a stock’s performance based on beta only: In [88]: sym = 'AMZN.O' In [89]: res[res['symbol'] == sym].corr() Out[89]: mu_capm mu_real mu_capm 1.000000 -0.004826 mu_real -0.004826 1.000000 In [90]: res[res['symbol'] == sym].plot(kind='bar', figsize=(10, 6), title=sym);

132

|

Chapter 4: Data-Driven Finance

Figure 4-9. CAPM-predicted versus realized stock returns for a single stock Figure 4-10 compares the averages of the CAPM-predicted stock returns with the averages of the realized returns. Also here, the CAPM does not do a good job. What is easy to see is that the CAPM predictions do not vary that much on average for the stocks analyzed; they are between 12.2% and 14.4%. However, the realized average returns of the stocks show a high variability; these are between 9.4% and 29.2%. Market portfolio performance and beta alone obviously cannot account for the observed returns of the (technology) stocks: In [91]: grouped grouped Out[91]: symbol AAPL.O AMZN.O INTC.O MSFT.O

= res.groupby('symbol').mean() mu_capm

mu_real

0.110855 0.206158 0.128223 0.259395 0.117929 0.116180 0.120844 0.192655

In [92]: grouped.plot(kind='bar', figsize=(10, 6), title='Average Values');

Normative Theories Revisited

|

133

Figure 4-10. Average CAPM-predicted versus average realized stock returns for multiple stocks

Predictive Power of the CAPM The predictive power of the CAPM with regard to the future per‐ formance of stocks, relative to the market portfolio, is pretty low or even nonexistent for certain stocks. One of the reasons is probably the fact that the CAPM rests on the same central assumptions as MVP theory, namely that investors care about only the (expected) return and (expected) volatility of a portfolio and/or stock. From a modeling point of view, one can ask whether the single risk factor is enough to explain variability in stock returns or whether there might be a nonlinear relationship between a stock’s return and the market portfolio performance.

Arbitrage Pricing Theory The predictive power of the CAPM seems quite limited given the results from the previous numerical example. A valid question is whether the market portfolio perfor‐ mance alone is enough to explain variability in stock returns. The answer of the APT is no—there can be more (even many more) factors that together explain variability in stock returns. “Arbitrage Pricing Theory” on page 90 formally describes the frame‐ work of APT that also relies on a linear relationship between the factors and a stock’s return.

134

|

Chapter 4: Data-Driven Finance

The data-driven investor recognizes that the CAPM is not sufficient to reliably pre‐ dict a stock’s performance relative to the market portfolio performance. Therefore, the investor decides to add to the market portfolio three additional factors that might drive a stock’s performance: • Market volatility (as represented by the VIX index, .VIX) • Exchange rates (as represented by the EUR/USD rate, EUR=) • Commodity prices (as represented by the gold price, XAU=) The following Python code implements a simple APT approach by using the four factors in combination with multivariate regression to explain a stock’s future perfor‐ mance in relation to the factors: In [93]: factors = ['.SPX', '.VIX', 'EUR=', 'XAU='] In [94]: res = pd.DataFrame() In [95]: np.set_printoptions(formatter={'float': lambda x: f'{x:5.2f}'}) In [96]: for sym in rets.columns[:4]: print('\n' + sym) print(71 * '=') for year in range(2010, 2019): rets_ = rets.loc[f'{year}-01-01':f'{year}-12-31'] reg = np.linalg.lstsq(rets_[factors], rets_[sym], rcond=-1)[0] rets_ = rets.loc[f'{year + 1}-01-01':f'{year + 1}-12-31'] mu_apt = np.dot(rets_[factors].mean() * 252, reg) mu_real = rets_[sym].mean() * 252 res = res.append(pd.DataFrame({'symbol': sym, 'mu_apt': mu_apt, 'mu_real': mu_real}, index=[year + 1])) print('{} | fl: {} | mu_apt: {:6.3f} | mu_real: {:6.3f}' .format(year + 1, reg.round(2), mu_apt, mu_real))

The four factors The multivariate regression The APT-predicted return of the stock The realized return of the stock The preceding code provides the following output: AAPL.O ======================================================================= 2011 | fl: [ 0.91 -0.04 -0.35 0.12] | mu_apt: 0.011 | mu_real: 0.228 2012 | fl: [ 0.76 -0.02 -0.24 0.05] | mu_apt: 0.099 | mu_real: 0.275

Normative Theories Revisited

|

135

2013 2014 2015 2016 2017 2018 2019

| | | | | | |

fl: fl: fl: fl: fl: fl: fl:

[ [ [ [ [ [ [

1.67 0.04 -0.56 0.10] | mu_apt: 0.366 | mu_real: 0.053 0.53 -0.00 0.02 0.16] | mu_apt: 0.050 | mu_real: 0.320 1.07 0.02 0.25 0.01] | mu_apt: -0.038 | mu_real: -0.047 1.21 0.01 -0.14 -0.02] | mu_apt: 0.110 | mu_real: 0.096 1.10 0.01 -0.15 -0.02] | mu_apt: 0.170 | mu_real: 0.381 1.06 -0.03 -0.15 0.12] | mu_apt: -0.088 | mu_real: -0.071 1.37 0.01 -0.20 0.13] | mu_apt: 0.364 | mu_real: 0.621

MSFT.O ======================================================================= 2011 | fl: [ 0.98 0.01 0.02 -0.11] | mu_apt: -0.008 | mu_real: -0.072 2012 | fl: [ 0.82 0.00 -0.03 -0.01] | mu_apt: 0.103 | mu_real: 0.029 2013 | fl: [ 1.14 0.00 -0.07 -0.01] | mu_apt: 0.294 | mu_real: 0.337 2014 | fl: [ 1.28 0.05 0.04 0.07] | mu_apt: 0.149 | mu_real: 0.216 2015 | fl: [ 1.20 0.03 0.05 0.01] | mu_apt: -0.016 | mu_real: 0.178 2016 | fl: [ 1.44 0.03 -0.17 -0.02] | mu_apt: 0.127 | mu_real: 0.113 2017 | fl: [ 1.33 0.01 -0.14 0.00] | mu_apt: 0.216 | mu_real: 0.321 2018 | fl: [ 1.10 -0.02 -0.14 0.22] | mu_apt: -0.087 | mu_real: 0.172 2019 | fl: [ 1.51 0.01 -0.16 -0.02] | mu_apt: 0.378 | mu_real: 0.440 INTC.O ======================================================================= 2011 | fl: [ 1.17 0.01 0.05 -0.13] | mu_apt: -0.010 | mu_real: 0.142 2012 | fl: [ 1.03 0.04 0.01 0.03] | mu_apt: 0.122 | mu_real: -0.163 2013 | fl: [ 1.06 -0.01 -0.10 0.01] | mu_apt: 0.267 | mu_real: 0.230 2014 | fl: [ 0.96 0.02 0.36 -0.02] | mu_apt: 0.063 | mu_real: 0.335 2015 | fl: [ 0.93 -0.01 -0.09 0.02] | mu_apt: 0.001 | mu_real: -0.052 2016 | fl: [ 1.02 0.00 -0.05 0.06] | mu_apt: 0.099 | mu_real: 0.051 2017 | fl: [ 1.41 0.02 -0.18 0.03] | mu_apt: 0.226 | mu_real: 0.242 2018 | fl: [ 1.12 -0.01 -0.11 0.17] | mu_apt: -0.076 | mu_real: 0.017 2019 | fl: [ 1.50 0.01 -0.34 0.30] | mu_apt: 0.431 | mu_real: 0.243 AMZN.O ======================================================================= 2011 | fl: [ 1.02 -0.03 -0.18 -0.14] | mu_apt: -0.016 | mu_real: -0.039 2012 | fl: [ 0.98 -0.01 -0.17 -0.09] | mu_apt: 0.117 | mu_real: 0.374 2013 | fl: [ 1.07 -0.00 0.09 0.00] | mu_apt: 0.282 | mu_real: 0.464 2014 | fl: [ 1.54 0.03 0.01 -0.08] | mu_apt: 0.176 | mu_real: -0.251 2015 | fl: [ 1.26 -0.02 0.45 -0.11] | mu_apt: -0.044 | mu_real: 0.778 2016 | fl: [ 1.06 -0.00 -0.15 -0.04] | mu_apt: 0.099 | mu_real: 0.104 2017 | fl: [ 0.94 -0.02 0.12 -0.03] | mu_apt: 0.185 | mu_real: 0.446 2018 | fl: [ 0.90 -0.04 -0.25 0.28] | mu_apt: -0.085 | mu_real: 0.251 2019 | fl: [ 1.99 0.05 -0.37 0.12] | mu_apt: 0.506 | mu_real: 0.207

Figure 4-11 compares the APT-predicted returns for a stock and its realized stock returns over time. Compared to the single-factor CAPM, there seems to be hardly any improvement: In [97]: sym = 'AMZN.O' In [98]: res[res['symbol'] == sym].corr() Out[98]: mu_apt mu_real

136

|

Chapter 4: Data-Driven Finance

mu_apt 1.000000 -0.098281 mu_real -0.098281 1.000000 In [99]: res[res['symbol'] == sym].plot(kind='bar', figsize=(10, 6), title=sym);

Figure 4-11. APT-predicted versus realized stock returns for a stock The same picture arises in Figure 4-12, produced by the following snippet, which compares the averages for multiple stocks. Because there is hardly any variation in the average APT predictions, there are large average differences to the realized returns: In [100]: grouped grouped Out[100]: symbol AAPL.O AMZN.O INTC.O MSFT.O

= res.groupby('symbol').mean() mu_apt

mu_real

0.116116 0.206158 0.135528 0.259395 0.124811 0.116180 0.128441 0.192655

In [101]: grouped.plot(kind='bar', figsize=(10, 6), title='Average Values');

Of course, the selection of the risk factors is of paramount importance in this context. The data-driven investor decides to find out what risk factors are typically considered relevant ones for stocks. After studying the paper by Bender et al. (2013), the investor replaces the original risk factors with a new set. In particular, the investor chooses the set as presented in Table 4-3.

Normative Theories Revisited

|

137

Figure 4-12. Average APT-predicted versus average realized stock returns for multiple stocks Table 4-3. Risk factors for APT Factor

Description

Market

MSCI World Gross Return Daily USD (PUS = Price Return) .dMIWO00000GUS

Size

MSCI World Equal Weight Price Net Index EOD

.dMIWO0000ENUS

Volatility

MSCI World Minimum Volatility Net Return

.dMIWO0000YNUS

Value

MSCI World Value Weighted Gross (NUS for Net)

.dMIWO000PkGUS

Risk

MSCI World Risk Weighted Gross USD EOD

.dMIWO000PlGUS

Growth

MSCI World Quality Net Return USD

.MIWO0000vNUS

Momentum MSCI World Momentum Gross Index USD EOD

RIC

.dMIWO0000NGUS

The following Python code retrieves a respective data set from a remote location and visualizes the normalized time series data (see Figure 4-13). Already a brief look reveals that the time series seem to be highly positively correlated: In [102]: factors = pd.read_csv('http://hilpisch.com/aiif_eikon_eod_factors.csv', index_col=0, parse_dates=True) In [103]: (factors / factors.iloc[0]).plot(figsize=(10, 6));

Retrieves factors time series data Normalizes and plots the data

138

|

Chapter 4: Data-Driven Finance

Figure 4-13. Normalized factors time series data This impression is confirmed by the following calculation and the resulting correla‐ tion matrix for the factor returns. All correlation factors are about 0.75 or higher: In [104]: start = '2017-01-01' end = '2020-01-01' In [105]: retsd = rets.loc[start:end].copy() retsd.dropna(inplace=True) In [106]: retsf = np.log(factors / factors.shift(1)) retsf = retsf.loc[start:end] retsf.dropna(inplace=True) retsf = retsf.loc[retsd.index].dropna() In [107]: retsf.corr() Out[107]: market size volatility value risk growth \ market 1.000000 0.935867 0.845010 0.964124 0.947150 0.959038 size 0.935867 1.000000 0.791767 0.965739 0.983238 0.835477 volatility 0.845010 0.791767 1.000000 0.778294 0.865467 0.818280 value 0.964124 0.965739 0.778294 1.000000 0.958359 0.864222 risk 0.947150 0.983238 0.865467 0.958359 1.000000 0.858546 growth 0.959038 0.835477 0.818280 0.864222 0.858546 1.000000 momentum 0.928705 0.796420 0.819585 0.818796 0.825563 0.952956 momentum market 0.928705 size 0.796420 volatility 0.819585 value 0.818796 risk 0.825563

Normative Theories Revisited

|

139

growth momentum

0.952956 1.000000

Defines start and end dates for data selection Selects the relevant returns data sub-set Calculates and processes the log returns for the factors Shows the correlation matrix for the factors The following Python code derives factor loadings for the original stocks but with the new factors. They are derived from the first half of the data set and applied to predict the stock return for the second half given the performance of the single factors. The realized return is also calculated. Both time series are compared in Figure 4-14. As to be expected given the high correlation of the factors, the explanatory power of the APT approach is not much higher compared to the CAPM: In [108]: res = pd.DataFrame() In [109]: np.set_printoptions(formatter={'float': lambda x: f'{x:5.2f}'}) In [110]: split = int(len(retsf) * 0.5) for sym in rets.columns[:4]: print('\n' + sym) print(74 * '=') retsf_, retsd_ = retsf.iloc[:split], retsd.iloc[:split] reg = np.linalg.lstsq(retsf_, retsd_[sym], rcond=-1)[0] retsf_, retsd_ = retsf.iloc[split:], retsd.iloc[split:] mu_apt = np.dot(retsf_.mean() * 252, reg) mu_real = retsd_[sym].mean() * 252 res = res.append(pd.DataFrame({'mu_apt': mu_apt, 'mu_real': mu_real}, index=[sym,]), sort=True) print('fl: {} | apt: {:.3f} | real: {:.3f}' .format(reg.round(1), mu_apt, mu_real))

AAPL.O ========================================================================== fl: [ 2.30 2.80 -0.70 -1.40 -4.20 2.00 -0.20] | apt: 0.115 | real: 0.301 MSFT.O ========================================================================== fl: [ 1.50 0.00 0.10 -1.30 -1.40 0.80 1.00] | apt: 0.181 | real: 0.304 INTC.O ========================================================================== fl: [-3.10 1.60 0.40 1.30 -2.60 2.50 1.10] | apt: 0.186 | real: 0.118

140

|

Chapter 4: Data-Driven Finance

AMZN.O ========================================================================== fl: [ 9.10 3.30 -1.00 -7.10 -3.10 -1.80 1.20] | apt: 0.019 | real: 0.050 In [111]: res.plot(kind='bar', figsize=(10, 6));

Figure 4-14. APT-predicted returns based on typical factors compared to realized returns The data-driven investor is not willing to dismiss the APT completely. Therefore, an additional test might shed some more light on the explanatory power of APT. To this end, the factor loadings are used to test whether APT can explain movements of the stock price over time (correctly). And indeed, although APT does not predict the absolute performance correctly (it is off by 10+ percentage points), it predicts the direction of the stock price movement correctly in the majority of cases (see Figure 4-15). The correlation between the predicted and realized returns is also pretty high at around 85%. However, the analysis uses realized factor returns to generate the APT predictions—something, of course, not available in practice a day before the rel‐ evant trading day: In [112]: sym Out[112]: 'AMZN.O' In [113]: rets_sym = np.dot(retsf_, reg) In [114]: rets_sym = pd.DataFrame(rets_sym, columns=[sym + '_apt'], index=retsf_.index) In [115]: rets_sym[sym + '_real'] = retsd_[sym]

Normative Theories Revisited

|

141

In [116]: rets_sym.mean() * 252 Out[116]: AMZN.O_apt 0.019401 AMZN.O_real 0.050344 dtype: float64 In [117]: rets_sym.std() * 252 ** 0.5 Out[117]: AMZN.O_apt 0.270995 AMZN.O_real 0.307653 dtype: float64 In [118]: rets_sym.corr() Out[118]: AMZN.O_apt AMZN.O_apt 1.000000 AMZN.O_real 0.832218

AMZN.O_real 0.832218 1.000000

In [119]: rets_sym.cumsum().apply(np.exp).plot(figsize=(10, 6));

Predicts the daily stock price returns given the realized factor returns Stores the results in a DataFrame object and adds column and index data Adds the realized stock price returns to the DataFrame object Calculates the annualized returns Calculates the annualized volatility Calculates the correlation factor

Figure 4-15. APT-predicted performance and real performance over time (gross)

142

| Chapter 4: Data-Driven Finance

How accurately does APT predict the direction of the stock price movement given the realized factor returns? The following Python code shows that the accuracy score is a bit better than 75%: In [120]: rets_sym['same'] = (np.sign(rets_sym[sym + '_apt']) == np.sign(rets_sym[sym + '_real'])) In [121]: rets_sym['same'].value_counts() Out[121]: True 288 False 89 Name: same, dtype: int64 In [122]: rets_sym['same'].value_counts()[True] / len(rets_sym) Out[122]: 0.7639257294429708

Debunking Central Assumptions The previous section provides a number of numerical, real-world examples showing how popular normative financial theories might fail in practice. This section argues that one of the major reasons is that central assumptions of these popular financial theories are invalid; that is, they simply do not describe the reality of financial markets. The two assumptions analyzed are normally distributed returns and linear relationships.

Normally Distributed Returns As a matter of fact, only a normal distribution is completely specified through its first (expectation) and second moment (standard deviation).

Sample data sets For illustration, consider a randomly generated set of standard normally distributed numbers as generated by the following Python code.4 Figure 4-16 shows the typical bell shape of the resulting histogram: In [1]: import numpy as np import pandas as pd from pylab import plt, mpl np.random.seed(100) plt.style.use('seaborn') mpl.rcParams['savefig.dpi'] = 300 mpl.rcParams['font.family'] = 'serif' In [2]: N = 10000

4 Numbers generated by the random number generator of NumPy are pseudorandom numbers, although they are

referenced throughout the book as random numbers.

Debunking Central Assumptions

|

143

In [3]: snrn = np.random.standard_normal(N) snrn -= snrn.mean() snrn /= snrn.std() In [4]: round(snrn.mean(), 4) Out[4]: -0.0 In [5]: round(snrn.std(), 4) Out[5]: 1.0 In [6]: plt.figure(figsize=(10, 6)) plt.hist(snrn, bins=35);

Draws standard normally distributed random numbers Corrects the first moment (expectation) to 0.0 Corrects the second moment (standard deviation) to 1.0

Figure 4-16. Standard normally distributed random numbers Now consider a set of random numbers that share the same first and second moment values but have a completely different distribution than Figure 4-17 illustrates. Although the moments are the same, this distribution only consists of three discrete values: In [7]: numbers = np.ones(N) * 1.5 split = int(0.25 * N) numbers[split:3 * split] = -1 numbers[3 * split:4 * split] = 0 In [8]: numbers -= numbers.mean()

144

|

Chapter 4: Data-Driven Finance

numbers /= numbers.std() In [9]: round(numbers.mean(), 4) Out[9]: 0.0 In [10]: round(numbers.std(), 4) Out[10]: 1.0 In [11]: plt.figure(figsize=(10, 6)) plt.hist(numbers, bins=35);

A set of numbers with three discrete values only Corrects the first moment (expectation) to 0.0 Corrects the second moment (standard deviation) to 1.0

Figure 4-17. Distribution with first and second moment of 0.0 and 1.0, respectively

First and Second Moment The first and second moment of a probability distribution only describe a normal distribution completely. There are infinitely many other distributions that might share the first two moments with a normal distribution while being completely different.

In preparation for a test of real financial returns, consider the following Python func‐ tions that allow one to visualize data as a histogram and to add a probability density function (PDF) of a normal distribution with the first two moments of the data: In [12]: import math import scipy.stats as scs

Debunking Central Assumptions

|

145

import statsmodels.api as sm In [13]: def dN(x, mu, sigma): ''' Probability density function of a normal random variable x. ''' z = (x - mu) / sigma pdf = np.exp(-0.5 * z ** 2) / math.sqrt(2 * math.pi * sigma ** 2) return pdf In [14]: def return_histogram(rets, title=''): ''' Plots a histogram of the returns. ''' plt.figure(figsize=(10, 6)) x = np.linspace(min(rets), max(rets), 100) plt.hist(np.array(rets), bins=50, density=True, label='frequency') y = dN(x, np.mean(rets), np.std(rets)) plt.plot(x, y, linewidth=2, label='PDF') plt.xlabel('log returns') plt.ylabel('frequency/probability') plt.title(title) plt.legend()

Plots the histogram of the data Plots the PDF of the corresponding normal distribution Figure 4-18 shows how well the histogram approximates the PDF for the standard normally distributed random numbers: In [15]: return_histogram(snrn)

Figure 4-18. Histogram and PDF for standard normally distributed numbers 146

| Chapter 4: Data-Driven Finance

By contrast, Figure 4-19 illustrates that the PDF of the normal distribution has noth‐ ing to do with the data shown as a histogram: In [16]: return_histogram(numbers)

Figure 4-19. Histogram and normal PDF for discrete numbers Another way of comparing a normal distribution to data is the Quantile-Quantile (QQ) plot. As Figure 4-20 shows, for normally distributed numbers, the numbers them‐ selves lie (mostly) on a straight line in the Q-Q plane: In [17]: def return_qqplot(rets, title=''): ''' Generates a Q-Q plot of the returns. ''' fig = sm.qqplot(rets, line='s', alpha=0.5) fig.set_size_inches(10, 6) plt.title(title) plt.xlabel('theoretical quantiles') plt.ylabel('sample quantiles') In [18]: return_qqplot(snrn)

Debunking Central Assumptions

|

147

Figure 4-20. Q-Q plot for standard normally distributed numbers Again, the Q-Q plot as shown in Figure 4-21 for the discrete numbers looks com‐ pletely different to the one in Figure 4-20: In [19]: return_qqplot(numbers)

Figure 4-21. Q-Q plot for discrete numbers Finally, one can also use statistical tests to check whether a set of numbers is normally distributed or not.

148

|

Chapter 4: Data-Driven Finance

The following Python function implements three tests: • Test for normal skew. • Test for normal kurtosis. • Test for normal skew and kurtosis combined. A p-value below 0.05 is generally considered to be a counter-indicator for normality; that is, the hypothesis that the numbers are normally distributed is rejected. In that sense, as in the preceding figures, the p-values for the two data sets speak for themselves: In [20]: def print_statistics(rets): print('RETURN SAMPLE STATISTICS') print('---------------------------------------------') print('Skew of Sample Log Returns {:9.6f}'.format( scs.skew(rets))) print('Skew Normal Test p-value {:9.6f}'.format( scs.skewtest(rets)[1])) print('---------------------------------------------') print('Kurt of Sample Log Returns {:9.6f}'.format( scs.kurtosis(rets))) print('Kurt Normal Test p-value {:9.6f}'.format( scs.kurtosistest(rets)[1])) print('---------------------------------------------') print('Normal Test p-value {:9.6f}'.format( scs.normaltest(rets)[1])) print('---------------------------------------------') In [21]: print_statistics(snrn) RETURN SAMPLE STATISTICS --------------------------------------------Skew of Sample Log Returns 0.016793 Skew Normal Test p-value 0.492685 --------------------------------------------Kurt of Sample Log Returns -0.024540 Kurt Normal Test p-value 0.637637 --------------------------------------------Normal Test p-value 0.707334 --------------------------------------------In [22]: print_statistics(numbers) RETURN SAMPLE STATISTICS --------------------------------------------Skew of Sample Log Returns 0.689254 Skew Normal Test p-value 0.000000 --------------------------------------------Kurt of Sample Log Returns -1.141902 Kurt Normal Test p-value 0.000000 --------------------------------------------Normal Test p-value 0.000000 ---------------------------------------------

Debunking Central Assumptions

|

149

Real financial returns The following Python code retrieves EOD data from a remote source, as done earlier in the chapter, and calculates the log returns for all financial time series contained in the data set. Figure 4-22 shows that the log returns of the S&P 500 stock index repre‐ sented as a histogram show a much higher peak and fatter tails when compared to the normal PDF with the sample expectation and standard deviation. These two insights are stylized facts because they can be consistently observed for different financial instruments: In [23]: raw = pd.read_csv('http://hilpisch.com/aiif_eikon_eod_data.csv', index_col=0, parse_dates=True).dropna() In [24]: rets = np.log(raw / raw.shift(1)).dropna() In [25]: symbol = '.SPX' In [26]: return_histogram(rets[symbol].values, symbol)

Figure 4-22. Frequency distribution and normal PDF for S&P 500 log returns Similar insights can be gained when considering the Q-Q plot for the S&P 500 log returns in Figure 4-23. In particular, the Q-Q plot visualizes the fat tails pretty well (points below the straight line to the left and above the straight line to the right): In [27]: return_qqplot(rets[symbol].values, symbol)

150

|

Chapter 4: Data-Driven Finance

Figure 4-23. Q-Q for S&P 500 log returns The Python code that follows conducts the statistical tests regarding the normality of the real financial returns for a selection of the financial time series from the data set. Real financial returns regularly fail such tests. Therefore, it is safe to conclude that the normality assumption about financial returns hardly, if at all, describes financial reality: In [28]: symbols = ['.SPX', 'AMZN.O', 'EUR=', 'GLD'] In [29]: for sym in symbols: print('\n{}'.format(sym)) print(45 * '=') print_statistics(rets[sym].values) .SPX ============================================= RETURN SAMPLE STATISTICS --------------------------------------------Skew of Sample Log Returns -0.497160 Skew Normal Test p-value 0.000000 --------------------------------------------Kurt of Sample Log Returns 4.598167 Kurt Normal Test p-value 0.000000 --------------------------------------------Normal Test p-value 0.000000 --------------------------------------------AMZN.O ============================================= RETURN SAMPLE STATISTICS

Debunking Central Assumptions

|

151

--------------------------------------------Skew of Sample Log Returns 0.135268 Skew Normal Test p-value 0.005689 --------------------------------------------Kurt of Sample Log Returns 7.344837 Kurt Normal Test p-value 0.000000 --------------------------------------------Normal Test p-value 0.000000 --------------------------------------------EUR= ============================================= RETURN SAMPLE STATISTICS --------------------------------------------Skew of Sample Log Returns -0.053959 Skew Normal Test p-value 0.268203 --------------------------------------------Kurt of Sample Log Returns 1.780899 Kurt Normal Test p-value 0.000000 --------------------------------------------Normal Test p-value 0.000000 --------------------------------------------GLD ============================================= RETURN SAMPLE STATISTICS --------------------------------------------Skew of Sample Log Returns -0.581025 Skew Normal Test p-value 0.000000 --------------------------------------------Kurt of Sample Log Returns 5.899701 Kurt Normal Test p-value 0.000000 --------------------------------------------Normal Test p-value 0.000000 ---------------------------------------------

Normality Assumption Although the normality assumption is a good approximation for many real-world phenomena, such as in physics, it is not appropri‐ ate and can even be dangerous when it comes to financial returns. Almost no financial return sample data set passes statistical nor‐ mality tests. Beyond the fact that it has proven useful in other domains, a major reason why this assumption is found in so many financial models is that it leads to elegant and relatively simple mathematical models, calculations, and proofs.

152

|

Chapter 4: Data-Driven Finance

Linear Relationships Similar to the “omnipresence” of the normality assumption in financial models and theories, linear relationships between variables seem to be another widespread bench‐ mark. This sub-section considers an important one, namely the assumed linear rela‐ tionship in the CAPM between the beta of a stock and its expected (realized) return. Generally speaking, the higher the beta is, the higher the expected return given a pos‐ itive market performance will be—in a fixed proportional way as given by the beta value itself. Recall the calculation of the betas, the CAPM expected returns, and the realized returns for a selection of technology stocks from the previous section, which is repeated in the following Python code for convenience. This time, the beta values are added to the results’ DataFrame object as well. In [30]: r = 0.005 In [31]: market = '.SPX' In [32]: res = pd.DataFrame() In [33]: for sym in rets.columns[:4]: for year in range(2010, 2019): rets_ = rets.loc[f'{year}-01-01':f'{year}-12-31'] muM = rets_[market].mean() * 252 cov = rets_.cov().loc[sym, market] var = rets_[market].var() beta = cov / var rets_ = rets.loc[f'{year + 1}-01-01':f'{year + 1}-12-31'] muM = rets_[market].mean() * 252 mu_capm = r + beta * (muM - r) mu_real = rets_[sym].mean() * 252 res = res.append(pd.DataFrame({'symbol': sym, 'beta': beta, 'mu_capm': mu_capm, 'mu_real': mu_real}, index=[year + 1]), sort=True)

The following analysis calculates the R2 score for a linear regression for which the beta is the independent variable and the expected CAPM return, given the market portfolio performance, is the dependent variable. R2 refers to the coefficient of deter‐ mination and measures how well a model performs compared to a baseline predictor in the form of a simple mean value. The linear regression can only explain around 10% of the variability in the expected CAPM return, a pretty low value, which is also confirmed through Figure 4-24: In [34]: from sklearn.metrics import r2_score

Debunking Central Assumptions

|

153

In [35]: reg = np.polyfit(res['beta'], res['mu_capm'], deg=1) res['mu_capm_ols'] = np.polyval(reg, res['beta']) In [36]: r2_score(res['mu_capm'], res['mu_capm_ols']) Out[36]: 0.09272355783573516 In [37]: res.plot(kind='scatter', x='beta', y='mu_capm', figsize=(10, 6)) x = np.linspace(res['beta'].min(), res['beta'].max()) plt.plot(x, np.polyval(reg, x), 'g--', label='regression') plt.legend();

Figure 4-24. Expected CAPM return versus beta (including linear regression) For the realized return, the explanatory power of the linear regression is even lower, with about 4.5% (see Figure 4-25). The linear regressions recover the positive rela‐ tionship between beta and stock returns—“the higher the beta, the higher the return given the (positive) market portfolio performance”—as indicated by the positive slope of the regression lines. However, they only explain a small part of the observed overall variability in the stock returns: In [38]: reg = np.polyfit(res['beta'], res['mu_real'], deg=1) res['mu_real_ols'] = np.polyval(reg, res['beta']) In [39]: r2_score(res['mu_real'], res['mu_real_ols']) Out[39]: 0.04466919444752959 In [40]: res.plot(kind='scatter', x='beta', y='mu_real', figsize=(10, 6)) x = np.linspace(res['beta'].min(), res['beta'].max()) plt.plot(x, np.polyval(reg, x), 'g--', label='regression') plt.legend();

154

|

Chapter 4: Data-Driven Finance

Figure 4-25. Expected CAPM return versus beta (including linear regression)

Linear Relationships As with the normality assumptions, linear relationships can often be observed in the physical world. However, in finance there are hardly any cases in which variables depend on each other in a clearly linear way. From a modeling point of view, linear relation‐ ships lead, as does the normality assumption, to elegant and rela‐ tively simple mathematical models, calculations, and proofs. In addition, the standard tool in financial econometrics, OLS regres‐ sion, is well suited to dealing with linear relationships in data. These are major reasons why normality and linearity are often deliberately chosen as convenient building blocks of financial models and theories.

Conclusions Science has been driven for centuries by the rigorous generation and analysis of data. However, finance used to be characterized by normative theories based on simplified mathematical models of the financial markets, relying on assumptions such as nor‐ mality of returns and linear relationships. The almost universal and comprehensive availability of (financial) data has led to a shift in focus from a theory-first approach to data-driven finance. Several examples based on real financial data illustrate that many popular financial models and theories cannot survive a confrontation with financial market realities. Although elegant, they might be too simplistic to capture the complexities, changing nature, and nonlinearities of financial markets.

Conclusions

|

155

References Books and papers cited in this chapter: Allais, M. 1953. “Le Comportement de l’Homme Rationnel devant le Risque: Critique des Postulats et Axiomes de l’Ecole Americaine.” Econometrica 21 (4): 503-546. Alexander, Carol. 2008a. Quantitative Methods in Finance. Market Risk Analysis I, West Sussex: John Wiley & Sons. ⸻. 2008b. Practical Financial Econometrics. Market Risk Analysis II, West Sus‐ sex: John Wiley & Sons. Bender, Jennifer et al. 2013. “Foundations of Factor Investing.” MSCI Research Insight. http://bit.ly/aiif_factor_invest. Campbell, John Y. 2018. Financial Decisions and Markets: A Course in Asset Pricing. Princeton and Oxford: Princeton University Press. Ellsberg, Daniel. 1961. “Risk, Ambiguity, and the Savage Axioms.” Quarterly Journal of Economics 75 (4): 643-669. Fontaine, Philippe and Robert Leonard. 2005. The Experiment in the History of Eco‐ nomics. London and New York: Routledge. Kopf, Dan. 2015. “The Discovery of Statistical Regression.” Priceonomics, November 6, 2015. http://bit.ly/aiif_ols. Lee, Kai-Fu. 2018. AI Superpowers: China, Silicon Valley, and the New World Order. Boston and New York: Houghton Mifflin Harcourt. Sapolsky, Robert M. 2018. Behave: The Biology of Humans at Our Best and Worst. New York: Penguin Books. Savage, Leonard J. (1954) 1972. The Foundations of Statistics. 2nd ed. New York: Dover Publications. Wigglesworth, Robin. 2019. “How Investment Analysts Became Data Miners.” Finan‐ cial Times, November 28, 2019. https://oreil.ly/QJGtd.

Python Code The following Python file contains a number of helper functions to simplify certain tasks in NLP: # # # # # # #

NLP Helper Functions Artificial Intelligence in Finance (c) Dr Yves J Hilpisch The Python Quants GmbH

156

|

Chapter 4: Data-Driven Finance

import re import nltk import string import pandas as pd from pylab import plt from wordcloud import WordCloud from nltk.corpus import stopwords from nltk.corpus import wordnet as wn from lxml.html.clean import Cleaner from sklearn.feature_extraction.text import TfidfVectorizer plt.style.use('seaborn') cleaner = Cleaner(style=True, links=True, allow_tags=[''], remove_unknown_tags=False) stop_words = stopwords.words('english') stop_words.extend(['new', 'old', 'pro', 'open', 'menu', 'close'])

def remove_non_ascii(s): ''' Removes all non-ascii characters. ''' return ''.join(i for i in s if ord(i) < 128) def clean_up_html(t): t = cleaner.clean_html(t) t = re.sub('[\n\t\r]', ' ', t) t = re.sub(' +', ' ', t) t = re.sub('', '', t) t = remove_non_ascii(t) return t def clean_up_text(t, numbers=False, punctuation=False): ''' Cleans up a text, e.g. HTML document, from HTML tags and also cleans up the text body. ''' try: t = clean_up_html(t) except: pass t = t.lower() t = re.sub(r"what's", "what is ", t) t = t.replace('(ap)', '') t = re.sub(r"\'ve", " have ", t) t = re.sub(r"can't", "cannot ", t) t = re.sub(r"n't", " not ", t) t = re.sub(r"i'm", "i am ", t) t = re.sub(r"\'s", "", t) t = re.sub(r"\'re", " are ", t) t = re.sub(r"\'d", " would ", t) t = re.sub(r"\'ll", " will ", t)

Python Code

|

157

t = re.sub(r'\s+', ' ', t) t = re.sub(r"\\", "", t) t = re.sub(r"\'", "", t) t = re.sub(r"\"", "", t) if numbers: t = re.sub('[^a-zA-Z ?!]+', '', t) if punctuation: t = re.sub(r'\W+', ' ', t) t = remove_non_ascii(t) t = t.strip() return t def nltk_lemma(word): ''' If one exists, returns the lemma of a word. I.e. the base or dictionary version of it. ''' lemma = wn.morphy(word) if lemma is None: return word else: return lemma def tokenize(text, min_char=3, lemma=True, stop=True, numbers=False): ''' Tokenizes a text and implements some transformations. ''' tokens = nltk.word_tokenize(text) tokens = [t for t in tokens if len(t) >= min_char] if numbers: tokens = [t for t in tokens if t[0].lower() in string.ascii_lowercase] if stop: tokens = [t for t in tokens if t not in stop_words] if lemma: tokens = [nltk_lemma(t) for t in tokens] return tokens def generate_word_cloud(text, no, name=None, show=True): ''' Generates a word cloud bitmap given a text document (string). It uses the Term Frequency (TF) and Inverse Document Frequency (IDF) vectorization approach to derive the importance of a word -- represented by the size of the word in the word cloud. Parameters ========== text: str text as the basis no: int

158

|

Chapter 4: Data-Driven Finance

number of words to be included name: str path to save the image show: bool whether to show the generated image or not ''' tokens = tokenize(text) vec = TfidfVectorizer(min_df=2, analyzer='word', ngram_range=(1, 2), stop_words='english' ) vec.fit_transform(tokens) wc = pd.DataFrame({'words': vec.get_feature_names(), 'tfidf': vec.idf_}) words = ' '.join(wc.sort_values('tfidf', ascending=True)['words'].head(no)) wordcloud = WordCloud(max_font_size=110, background_color='white', width=1024, height=768, margin=10, max_words=150).generate(words) if show: plt.figure(figsize=(10, 10)) plt.imshow(wordcloud, interpolation='bilinear') plt.axis('off') plt.show() if name is not None: wordcloud.to_file(name) def generate_key_words(text, no): try: tokens = tokenize(text) vec = TfidfVectorizer(min_df=2, analyzer='word', ngram_range=(1, 2), stop_words='english' ) vec.fit_transform(tokens) wc = pd.DataFrame({'words': vec.get_feature_names(), 'tfidf': vec.idf_}) words = wc.sort_values('tfidf', ascending=False)['words'].values words = [ a for a in words if not a.isnumeric()][:no] except: words = list() return words

Python Code

|

159

CHAPTER 5

Machine Learning

Dataism says that the universe consists of data flows, and the value of any phenom‐ enon or entity is determined by its contribution to data processing….Dataism thereby collapses the barrier between animals [humans] and machines, and expects electronic algorithms to eventually decipher and outperform biochemical algorithms. —Yuval Noah Harari (2015) Machine learning is the scientific method on steroids. It follows the same process of generating, testing, and discarding or refining hypotheses. But while a scientist may spend his or her whole life coming up with and testing a few hundred hypotheses, a machine learning system can do the same in a second. Machine learning automates discovery. It’s no surprise, then, that it’s revolutionizing science as much as it’s revolu‐ tionizing business. —Pedro Domingos (2015)

This chapter is about machine learning as a process. Although it uses specific algo‐ rithms and specific data for illustration, the notions and approaches discussed in this chapter are general in nature. The goal is to present the most important elements of machine learning in a single place and in an easy-to-understand and easy-to-visualize manner. The approach of this chapter is practical and illustrative in nature, omitting most technical details throughout. In that sense, the chapter provides a kind of blue‐ print for later, more realistic machine learning applications. “Learning” on page 162 briefly discusses the very notion of a machine that learns. “Data” on page 162 imports and preprocesses the sample data used in later sections. The sample data is based on a time series for the EUR/USD exchange rate. “Success” on page 165 implements OLS regression and neural network estimation given the sample data and uses the mean-squared error as the measure of success. “Capacity” on page 169 discusses the role of the model capacity in making models more successful in the context of estimation problems. “Evaluation” on page

161

172 explains the role that model evaluation, typically based on a validation data subset, plays in the machine-learning process. “Bias and Variance” on page 178 discusses the notions of high bias and high variance models and their typical characteristics in the context of estimation problems. “Cross-Validation” on page 180 illustrates the concept of cross-validation to avoid, among other things, overfitting due to a toolarge model capacity. VanderPlas (2017, ch. 5) discusses topics similar to the ones covered in this chapter, making use primarily of the scikit-learn Python package. Chollet (2017, ch. 4) also provides an overview similar to the one provided here, but primarily makes use of the Keras deep learning package. Goodfellow et al. (2016, ch. 5) give a more technical and mathematical overview of machine learning and related important concepts.

Learning On a formal, more abstract level, learning by an algorithm or computer program can be defined as in Mitchell (1997): A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T , as measured by P, improves with experience E.

There is a class of tasks that are to be performed (for example, estimation or classifica‐ tion). Then there is a performance measure, such as the mean-squared error (MSE) or the accuracy ratio. Then there is learning as measured by the improvement in perfor‐ mance given the experience of the algorithm with the task. The class of tasks at hand is described in general based on the given data set, which includes the features data and the labels data in the case of supervised learning, or only the features data in the case of unsupervised learning.

Learning Task Versus Task to Learn In the definition of learning through an algorithm or computer program, it is important to note the difference between the task of learning and the tasks to be learned. Learning means to learn how to (best) execute a certain task, such as estimation or classification.

Data This section introduces the sample data set to be used in the sections to follow. The sample data is created based on a real financial time series for the EUR/USD exchange rate. First, the data is imported from a CSV file, and then the data is resam‐ pled to monthly data and stored in a Series object: In [1]: import numpy as np import pandas as pd

162

|

Chapter 5: Machine Learning

from pylab import plt, mpl np.random.seed(100) plt.style.use('seaborn') mpl.rcParams['savefig.dpi'] = 300 mpl.rcParams['font.family'] = 'serif' In [2]: url = 'http://hilpisch.com/aiif_eikon_eod_data.csv' In [3]: raw = pd.read_csv(url, index_col=0, parse_dates=True)['EUR='] In [4]: raw.head() Out[4]: Date 2010-01-01 1.4323 2010-01-04 1.4411 2010-01-05 1.4368 2010-01-06 1.4412 2010-01-07 1.4318 Name: EUR=, dtype: float64 In [5]: raw.tail() Out[5]: Date 2019-12-26 1.1096 2019-12-27 1.1175 2019-12-30 1.1197 2019-12-31 1.1210 2020-01-01 1.1210 Name: EUR=, dtype: float64 In [6]: l = raw.resample('1M').last() In [7]: l.plot(figsize=(10, 6), title='EUR/USD monthly');

Imports the financial time series data Resamples the data to monthly time intervals Figure 5-1 shows the financial time series.

Data

|

163

Figure 5-1. EUR/USD exchange rate as time series (monthly) To have a single feature only, the following Python code creates a synthetic feature vector. This allows for simple visualizations in two dimensions. The synthetic feature (independent variable), of course, does not have any explanatory power for the EUR/USD exchange rate (labels data, dependent variable). In what follows, it is also abstracted from the fact that the labels data is sequential and temporal in nature. The sample data set is treated in this chapter as a general data set composed of a onedimensional features vector and a one-dimensional labels vector. Figure 5-2 visualizes the sample data set that implies an estimation problem is the task at hand: In [8]: l = l.values l -= l.mean() In [9]: f = np.linspace(-2, 2, len(l)) In [10]: plt.figure(figsize=(10, 6)) plt.plot(f, l, 'ro') plt.title('Sample Data Set') plt.xlabel('features') plt.ylabel('labels');

Transforms the labels data to an ndarray object Subtracts the mean value from the data element-wise Creates a synthetic feature as an ndarray object

164

|

Chapter 5: Machine Learning

Figure 5-2. Sample data set

Success The measure of success for estimation problems in general is the MSE, as used in Chapter 1. Based on the MSE, success is judged given the labels data as the relevant benchmark and the predicted values of an algorithm after having been exposed to the data set or parts of it. As in Chapter 1, two algorithms are considered in this and the following sections: OLS regression and neural networks. First is OLS regression. The application is straightforward, as the following Python code illustrates. The regression result is shown in Figure 5-3 for a regression includ‐ ing monomials up to the fifth order. The resulting MSE is also calculated: In [11]: def MSE(l, p): return np.mean((l - p) ** 2) In [12]: reg = np.polyfit(f, l, deg=5) reg Out[12]: array([-0.01910626, -0.0147182 , -0.03275423])

0.10990388,

0.06007211, -0.20833598,

In [13]: p = np.polyval(reg, f) In [14]: MSE(l, p) Out[14]: 0.0034166422957371025 In [15]: plt.figure(figsize=(10, 6)) plt.plot(f, l, 'ro', label='sample data') plt.plot(f, p, '--', label='regression') plt.legend();

Success

|

165

The function MSE calculates the mean-squared error. The fitting of the OLS regression model up to and including fifth-order mono‐ mials. The prediction by the OLS regression model given the optimal parameters. The MSE value given the prediction values.

Figure 5-3. Sample data and cubic regression line OLS regression is generally solved analytically. Therefore, no iterated learning takes place. However, one can simulate a learning procedure by gradually exposing the algorithm to more data. The following Python code implements OLS regression and prediction, starting with a few samples only and gradually increasing the number to finally reach the complete length of the data set. The regression step is implemented based on the smaller sub-sets, whereas the prediction steps are implemented based on the whole features data in each case. In general, the MSE drops significantly when increasing the training data set: In [16]: for i in range(10, len(f) + 1, 20): reg = np.polyfit(f[:i], l[:i], deg=3) p = np.polyval(reg, f) mse = MSE(l, p) print(f'{i:3d} | MSE={mse}') 10 | MSE=248628.10681642237 30 | MSE=731.9382249304651 50 | MSE=12.236088505004465 70 | MSE=0.7410590619743301 90 | MSE=0.0057430617304093275 110 | MSE=0.006492800939555582

166

|

Chapter 5: Machine Learning

Regression step based on data sub-set Prediction step based on the complete data set Resulting MSE value Second is the neural network. The application to the sample data is again straightfor‐ ward and similar to the case in Chapter 1. Figure 5-4 shows how the neural network approximates the sample data: In [17]: import tensorflow as tf tf.random.set_seed(100) In [18]: from keras.layers import Dense from keras.models import Sequential Using TensorFlow backend. In [19]: model = Sequential() model.add(Dense(256, activation='relu', input_dim=1)) model.add(Dense(1, activation='linear')) model.compile(loss='mse', optimizer='rmsprop') In [20]: model.summary() Model: "sequential_1" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense_1 (Dense) (None, 256) 512 _________________________________________________________________ dense_2 (Dense) (None, 1) 257 ================================================================= Total params: 769 Trainable params: 769 Non-trainable params: 0 _________________________________________________________________ In [21]: %time model.fit(f, l, epochs=1500, verbose=False) CPU times: user 5.89 s, sys: 761 ms, total: 6.66 s Wall time: 4.43 s Out[21]: In [22]: p = model.predict(f).flatten() In [23]: MSE(l, p) Out[23]: 0.0020217512014360102 In [24]: plt.figure(figsize=(10, 6)) plt.plot(f, l, 'ro', label='sample data') plt.plot(f, p, '--', label='DNN approximation') plt.legend();

Success

|

167

The neural network is a shallow network with a single hidden layer. The fitting step with a relatively high number of epochs. The prediction step that also flattens the ndarray object. The resulting MSE value for the DNN prediction.

Figure 5-4. Sample data and neural network approximation With the Keras package, the MSE values are stored after every learning step. Figure 5-5 shows how the MSE value (“loss”) decreases on average (as far as one can tell from the plot) with the increasing number of epochs over which the neural net‐ work is trained: In [25]: import pandas as pd In [26]: res = pd.DataFrame(model.history.history) In [27]: res.tail() Out[27]: loss 1495 0.001547 1496 0.001520 1497 0.001456 1498 0.001356 1499 0.001325 In [28]: res.iloc[100:].plot(figsize=(10, 6)) plt.ylabel('MSE') plt.xlabel('epochs');

168

|

Chapter 5: Machine Learning

Figure 5-5. MSE values against number of training epochs

Capacity The capacity of a model or algorithm defines what types of functions or relationships the model or algorithm can basically learn. In the case of OLS regression based on monomials only, there is only one parameter that defines the capacity of the model: the degree of the highest monomial to be used. If this degree parameter is set to deg=3, the OLS regression model can learn functional relationships of constant, lin‐ ear, quadratic, or cubic type. The higher the parameter deg is, the higher the capacity of the OLS regression model will be. The following Python code starts at deg=1 and increases the degree in increments of two. The MSE values monotonically decrease with the increasing degree parameter. Figure 5-6 shows the regression lines for all degrees considered: In [29]: reg = {} for d in range(1, 12, 2): reg[d] = np.polyfit(f, l, deg=d) p = np.polyval(reg[d], f) mse = MSE(l, p) print(f'{d:2d} | MSE={mse}') 1 | MSE=0.005322474034260403 3 | MSE=0.004353110724143185 5 | MSE=0.0034166422957371025 7 | MSE=0.0027389501772354025 9 | MSE=0.001411961626330845 11 | MSE=0.0012651237868752322 In [30]: plt.figure(figsize=(10, 6)) plt.plot(f, l, 'ro', label='sample data')

Capacity

|

169

for d in reg: p = np.polyval(reg[d], f) plt.plot(f, p, '--', label=f'deg={d}') plt.legend();

Regression step for different values for deg

Figure 5-6. Regression lines for different highest degrees The capacity of a neural network depends on a number of hyperparameters. Among them are, in general, the following: • Number of hidden layers • Number of hidden units for each hidden layer Together, these two hyperparameters define the number of trainable parameters (weights) in the neural network. The neural network model in the previous section has a relatively low number of trainable parameters. Adding, for example, just one more layer of the same size increases the number of trainable parameters signifi‐ cantly. Although the number of training epochs may need to be increased, the MSE value decreases significantly for the neural network with the higher capacity, and the fit also seems much better visually, as Figure 5-7 shows: In [31]: def create_dnn_model(hl=1, hu=256): ''' Function to create Keras DNN model. Parameters ========== hl: int number of hidden layers hu: int

170

|

Chapter 5: Machine Learning

number of hidden units (per layer) ''' model = Sequential() for _ in range(hl): model.add(Dense(hu, activation='relu', input_dim=1)) model.add(Dense(1, activation='linear')) model.compile(loss='mse', optimizer='rmsprop') return model In [32]: model = create_dnn_model(3) In [33]: model.summary() Model: "sequential_2" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense_3 (Dense) (None, 256) 512 _________________________________________________________________ dense_4 (Dense) (None, 256) 65792 _________________________________________________________________ dense_5 (Dense) (None, 256) 65792 _________________________________________________________________ dense_6 (Dense) (None, 1) 257 ================================================================= Total params: 132,353 Trainable params: 132,353 Non-trainable params: 0 _________________________________________________________________ In [34]: %time model.fit(f, l, epochs=2500, verbose=False) CPU times: user 34.9 s, sys: 5.91 s, total: 40.8 s Wall time: 15.5 s Out[34]: In [35]: p = model.predict(f).flatten() In [36]: MSE(l, p) Out[36]: 0.00046612284916401614 In [37]: plt.figure(figsize=(10, 6)) plt.plot(f, l, 'ro', label='sample data') plt.plot(f, p, '--', label='DNN approximation') plt.legend();

Adds potentially many layers to the neural network A deep neural network with three hidden layers The summary shows the increased number of trainable parameters (increased capacity) Capacity

|

171

Figure 5-7. Sample data and DNN approximation (higher capacity)

Evaluation In the previous sections, the analysis focuses on the performance of estimation algo‐ rithms on the sample data set as a whole. As a general rule, the capacity of the model or algorithm directly influences its performance when training and evaluating it on the same data set. However, this is the “simple and easy case” in ML. The more com‐ plex and interesting case is when a trained model or algorithm shall be used for a generalization on data that the model or algorithm has not seen before. Such a gener‐ alization can, for example, be the prediction (estimation) of a future stock price, given the history of stock prices, or the classification of potential debtors as “creditworthy” or “not creditworthy,” given the data from existing debtors. Although the term prediction is often used freely in the context of estimations, given the features data set used for training, a real prediction probably entails predicting something not known up front and never seen before. Again, the prediction of a future stock price is a good example for a real prediction in a temporal sense. In general, a given data set is divided into sub-sets that each have different purposes: Training data set This is the sub-set used for the training of the algorithm. Validation data set This is the sub-set used for validating the performance of the algorithm during training—and this data set is different from the training data set.

172

| Chapter 5: Machine Learning

Test data set This is the sub-set on which the trained algorithm is only tested after the training is finished. Insights that are gained by applying a (currently) trained algorithm on the validation data set might reflect on the training itself (for example, by adjusting the hyperpara‐ meters of a model). On the other hand, the idea is that insights from testing the trained algorithm on the test data set shall not be reflected in the training itself or the hyperparameters. The following Python code chooses, somewhat arbitrarily, 25% of the sample data for testing; the model or algorithm will not see this data before the training (learning) is finished. Similarly, 25% of the sample data is reserved for validation; this data is used to monitor performance during the training step and possibly during many learning iterations. The remaining 50% is used for the training (learning) itself.1 Given the sample data set, it makes sense to apply shuffling techniques to populate all sample data sub-sets randomly: In [38]: te = int(0.25 * len(f)) va = int(0.25 * len(f)) In [39]: np.random.seed(100) ind = np.arange(len(f)) np.random.shuffle(ind) In [40]: ind_te = np.sort(ind[:te]) ind_va = np.sort(ind[te:te + va]) ind_tr = np.sort(ind[te + va:]) In [41]: f_te = f[ind_te] f_va = f[ind_va] f_tr = f[ind_tr] In [42]: l_te = l[ind_te] l_va = l[ind_va] l_tr = l[ind_tr]

Number of test data set samples Number of validation data set samples Randomized index for complete data set Resulting sorted indexes for the data sub-sets

1 Often, the rule of thumb mentioned in this context is “60%, 20%, 20%” for the split of a given data set into

training, validation, and testing data sub-sets.

Evaluation

|

173

Resulting features data sub-sets Resulting labels data sub-sets

Randomized Sampling The randomized population of training, validation, and test data sets is a common and useful technique for data sets that are neither sequence-like nor temporal in nature. However, when one is deal‐ ing, say, with a financial time series, shuffling the data is generally to be avoided because it breaks up temporal structures and sneaks foresight bias into the process by using, for example, later samples for training and implementing the testing on earlier samples.

Based on the training and validation data sub-sets, the following Python code imple‐ ments a regression for different deg parameter values and calculates the MSE values for the predictions on both data sub-sets. Although the MSE values on the training data set decrease monotonically, the MSE values on the validation data set often reach a minimum for a certain parameter value and then increase again. This phenomenon indicates what is called overfitting. Figure 5-8 shows the regression fits for the differ‐ ent values of deg and compares the fits for both the training data and validation data sets: In [43]: reg = {} mse = {} for d in range(1, 22, 4): reg[d] = np.polyfit(f_tr, l_tr, deg=d) p = np.polyval(reg[d], f_tr) mse_tr = MSE(l_tr, p) p = np.polyval(reg[d], f_va) mse_va = MSE(l_va, p) mse[d] = (mse_tr, mse_va) print(f'{d:2d} | MSE_tr={mse_tr:7.5f} | MSE_va={mse_va:7.5f}') 1 | MSE_tr=0.00574 | MSE_va=0.00492 5 | MSE_tr=0.00375 | MSE_va=0.00273 9 | MSE_tr=0.00132 | MSE_va=0.00243 13 | MSE_tr=0.00094 | MSE_va=0.00183 17 | MSE_tr=0.00060 | MSE_va=0.00153 21 | MSE_tr=0.00046 | MSE_va=0.00837 In [44]: fig, ax = plt.subplots(2, 1, figsize=(10, 8), sharex=True) ax[0].plot(f_tr, l_tr, 'ro', label='training data') ax[1].plot(f_va, l_va, 'go', label='validation data') for d in reg: p = np.polyval(reg[d], f_tr) ax[0].plot(f_tr, p, '--', label=f'deg={d} (tr)') p = np.polyval(reg[d], f_va) plt.plot(f_va, p, '--', label=f'deg={d} (va)')

174

|

Chapter 5: Machine Learning

ax[0].legend() ax[1].legend();

MSE value for the training data set MSE value for the validation data set

Figure 5-8. Training and validation data including regression fits With Keras and the neural network model, the validation data set performance can be monitored for every single learning step. One can also use callback functions to stop the model training early when no further improvements, say, in the performance on the training data set, are observed. The following Python code makes use of such a callback function. Figure 5-9 shows the predictions of the neural network for the training and validation data sets: In [45]: from keras.callbacks import EarlyStopping In [46]: model = create_dnn_model(2, 256) In [47]: callbacks = [EarlyStopping(monitor='loss', patience=100, restore_best_weights=True)] In [48]: %%time model.fit(f_tr, l_tr, epochs=3000, verbose=False, validation_data=(f_va, l_va), callbacks=callbacks)

Evaluation

|

175

CPU times: user 8.07 s, sys: 1.33 s, total: 9.4 s Wall time: 4.81 s Out[48]: In [49]: fig, ax = plt.subplots(2, 1, sharex=True, figsize=(10, 8)) ax[0].plot(f_tr, l_tr, 'ro', label='training data') p = model.predict(f_tr) ax[0].plot(f_tr, p, '--', label=f'DNN (tr)') ax[0].legend() ax[1].plot(f_va, l_va, 'go', label='validation data') p = model.predict(f_va) ax[1].plot(f_va, p, '--', label=f'DNN (va)') ax[1].legend();

Learning is stopped based on training data MSE value. It is only stopped after a certain number of epochs that do not show an improvement. The best weights are restored when the learning is stopped. The validation data sub-sets are specified. The callback function is passed to the fit() method.

Figure 5-9. Training and validation data including DNN predictions

176

|

Chapter 5: Machine Learning

Keras allows analysis of the change in the MSE values on both data sets for every sin‐

gle epoch the model has been trained in. Figure 5-10 shows that the MSE values decrease with the increasing number of training epochs, although only on average and not monotonically: In [50]: res = pd.DataFrame(model.history.history) In [51]: res.tail() Out[51]: val_loss loss 1375 0.000854 0.000544 1376 0.000685 0.000473 1377 0.001326 0.000942 1378 0.001026 0.000867 1379 0.000710 0.000500 In [52]: res.iloc[35::25].plot(figsize=(10, 6)) plt.ylabel('MSE') plt.xlabel('epochs');

Figure 5-10. MSE values for DNN model on the training and validation data sets In the case of OLS regression, one would probably choose a high—but not too high— value for the degree parameter, such as deg=9. The parameterization of the neural network model automatically gives the best model configuration at the end of the training. Figure 5-10 compares the predictions of both models to each other and to the test data set. Given the nature of the sample data, the somewhat better test data set performance of the neural network should not come as a surprise: In [53]: p_ols = np.polyval(reg[5], f_te) p_dnn = model.predict(f_te).flatten() In [54]: MSE(l_te, p_ols)

Evaluation

|

177

Out[54]: 0.0038960346771028356 In [55]: MSE(l_te, p_dnn) Out[55]: 0.000705705678438721 In [56]: plt.figure(figsize=(10, 6)) plt.plot(f_te, l_te, 'ro', label='test data') plt.plot(f_te, p_ols, '--', label='OLS prediction') plt.plot(f_te, p_dnn, '-.', label='DNN prediction'); plt.legend();

Figure 5-11. Test data and predictions from OLS regression and the DNN model

Bias and Variance A major problem in ML in general and when applying ML algorithms to financial data in particular is the problem of overfitting. A model is overfitting its training data when the performance is worse on the validation and test data than on the training data. An example using OLS regression can illustrate the problem both visually and numerically. The following Python code uses smaller sub-sets for both training and validation and implements a linear regression, as well as one of higher order. The linear regression fit, as shown in Figure 5-12, has a high bias on the training data set; absolute differ‐ ences between predictions and labels data are relatively high. The higher-order fit shows a high variance. It hits all training data points exactly, but the fit itself varies significantly to achieve the perfect fit: In [57]: f_tr = f[:20:2] l_tr = l[:20:2] In [58]: f_va = f[1:20:2]

178

|

Chapter 5: Machine Learning

l_va = l[1:20:2] In [59]: reg_b = np.polyfit(f_tr, l_tr, deg=1) In [60]: reg_v = np.polyfit(f_tr, l_tr, deg=9, full=True)[0] In [61]: f_ = np.linspace(f_tr.min(), f_va.max(), 75) In [62]: plt.figure(figsize=(10, 6)) plt.plot(f_tr, l_tr, 'ro', label='training data') plt.plot(f_va, l_va, 'go', label='validation data') plt.plot(f_, np.polyval(reg_b, f_), '--', label='high bias') plt.plot(f_, np.polyval(reg_v, f_), '--', label='high variance') plt.ylim(-0.2) plt.legend(loc=2);

Smaller features data sub-set Smaller labels data sub-set High bias OLS regression (linear) High variance OLS regression (higher order) Enlarged features data set for plotting

Figure 5-12. High bias and high variance OLS regression fits Figure 5-12 shows that a high bias fit performs worse in the example than a high var‐ iance fit on the training data. But the high variance fit, which is overfitting here to a large extent, performs much worse on the validation data. This can be illustrated by

Bias and Variance

|

179

comparing performance measures for all cases. The following Python code calculates not only the MSE values, but also the R2 values: In [63]: from sklearn.metrics import r2_score In [64]: def evaluate(reg, f, l): p = np.polyval(reg, f) bias = np.abs(l - p).mean() var = p.var() msg = f'MSE={MSE(l, p):.4f} | R2={r2_score(l, p):9.4f} | ' msg += f'bias={bias:.4f} | var={var:.4f}' print(msg) In [65]: evaluate(reg_b, f_tr, l_tr) MSE=0.0026 | R2= 0.3484 | bias=0.0423 | var=0.0014 In [66]: evaluate(reg_b, f_va, l_va) MSE=0.0032 | R2= 0.4498 | bias=0.0460 | var=0.0014 In [67]: evaluate(reg_v, f_tr, l_tr) MSE=0.0000 | R2= 1.0000 | bias=0.0000 | var=0.0040 In [68]: evaluate(reg_v, f_va, l_va) MSE=0.8752 | R2=-149.2658 | bias=0.3565 | var=0.7539

Model bias as mean absolute differences Model variance as variance of model predictions Performance of high bias model on training data Performance of high bias model on validation data Performance of high variance model on training data Performance of high variance model on validation data The results show that performance of the high bias model is roughly comparable on both the training and validation data sets. By contrast, the performance of the high variance model is perfect on the training data and pretty bad on the validation data.

Cross-Validation A standard approach to avoid overfitting is cross-validation, during which multiple training and validation data populations are tested. The scikit-learn package pro‐ vides functionality to implement cross-validation in a standardized way. The function cross_val_score can be applied to any scikit-learn model object.

180

|

Chapter 5: Machine Learning

The following code implements the OLS regression approach on the complete sample data set, using a polynomial OLS regression model from scikit-learn. The five-fold cross-validation is implemented for different degrees for the highest polynomial. The cross-validation scores become, on average, worse the higher the highest degree is in the regression. Particularly bad results are observed when the first 20% of the data is used for validation (data on the left-hand side in Figure 5-3) or the final 20% of the data is used (data on the right-hand side in Figure 5-3). Similarly, the best validation scores are observed for the middle 20% of the sample data set: In [69]: from from from from

sklearn.model_selection import cross_val_score sklearn.preprocessing import PolynomialFeatures sklearn.linear_model import LinearRegression sklearn.pipeline import make_pipeline

In [70]: def PolynomialRegression(degree=None, **kwargs): return make_pipeline(PolynomialFeatures(degree), LinearRegression(**kwargs)) In [71]: np.set_printoptions(suppress=True, formatter={'float': lambda x: f'{x:12.2f}'}) In [72]: print('\nCross-validation scores') print(74 * '=') for deg in range(0, 10, 1): model = PolynomialRegression(deg) cvs = cross_val_score(model, f.reshape(-1, 1), l, cv=5) print(f'deg={deg} | ' + str(cvs.round(2))) Cross-validation scores ========================================================================== deg=0 | [ -6.07 -7.34 -0.09 -6.32 -8.69] deg=1 | [ -0.28 -1.40 0.16 -1.66 -4.62] deg=2 | [ -3.48 -2.45 0.19 -1.57 -12.94] deg=3 | [ -0.00 -1.24 0.32 -0.48 -43.62] deg=4 | [ -222.81 -2.88 0.37 -0.32 -496.61] deg=5 | [ -143.67 -5.85 0.49 0.12 -1241.04] deg=6 | [ -4038.96 -14.71 0.49 -0.33 -317.32] deg=7 | [ -9937.83 -13.98 0.64 0.22 -18725.61] deg=8 | [ -3514.36 -11.22 -0.15 -6.29 -298744.18] deg=9 | [ -7454.15 -0.91 0.15 -0.41 -13580.75]

Creates a polynomial regression model class Adjusts the default printing settings for numpy Implements the five-fold cross-validation Keras provides wrapper classes to use Keras model objects with scikit-learn func‐ tionality, such as the cross_val_score function. The following example uses the

Cross-Validation

|

181

KerasRegressor class to wrap the neural network models and to apply the crossvalidation to them. The cross-validation scores are better throughout for the two net‐ works tested when compared to the OLS regression cross-validation scores. The neural network capacity does not play too large a role in this example: In [73]: np.random.seed(100) tf.random.set_seed(100) from keras.wrappers.scikit_learn import KerasRegressor In [74]: model = KerasRegressor(build_fn=create_dnn_model, verbose=False, epochs=1000, hl=1, hu=36) In [75]: %time cross_val_score(model, f, l, cv=5) CPU times: user 18.6 s, sys: 2.17 s, total: 20.8 s Wall time: 14.6 s Out[75]: array([

-0.02, -0.01])

-0.01,

-0.00,

-0.00,

In [76]: model = KerasRegressor(build_fn=create_dnn_model, verbose=False, epochs=1000, hl=3, hu=256) In [77]: %time cross_val_score(model, f, l, cv=5) CPU times: user 1min 5s, sys: 11.6 s, total: 1min 16s Wall time: 30.1 s Out[77]: array([

-0.08, -0.05])

-0.00,

-0.00,

-0.00,

Wrapper class for neural network with low capacity Cross-validation for neural network with low capacity Wrapper class for neural network with high capacity Cross-validation for neural network with high capacity

Avoiding Overfitting Overfitting—when a model performs much better on a training data set than on the validation and test data sets—is to be avoided in ML in general and in finance in particular. Proper evaluation procedures and analyses, such as cross-validation, help in prevent‐ ing overfitting and in finding, for example, an adequate model capacity.

182

|

Chapter 5: Machine Learning

Conclusions This chapter presents a blueprint for a machine learning process. The main elements presented are as follows: Learning What exactly is meant by machine learning? Data What raw data and what (preprocessed) features and labels data is to be used? Success Given the problem as defined indirectly by the data (estimation, classification, etc.), what is the appropriate measure of success? Capacity Which role does the model capacity play, and what might be an adequate capacity given the problem at hand? Evaluation How shall the model performance be evaluated given the purpose of the trained model? Bias and variance Which models are better suited for the problem at hand: those with rather high bias or rather high variance? Cross-validation For non-sequence-like data sets, how does the model perform when crossvalidated on different configurations for the training and validation data sub-sets used? This blueprint is applied loosely in subsequent chapters to a number of real-world financial use cases. For more background information and details about machine learning as a process, refer to the references listed at the end of this chapter.

References Books and papers cited in this chapter: Chollet, François. 2017. Deep Learning with Python. Shelter Island: Manning. Domingos, Pedro. 2015. The Master Algorithm: How the Quest for the Ultimate Learn‐ ing Machine Will Remake Our World. New York: Basic Books. Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. Cam‐ bridge: MIT Press. http://deeplearningbook.org.

Conclusions

|

183

Harari, Yuval Noah. 2015. Homo Deus: A Brief History of Tomorrow. London: Harvill Secker. Mitchell, Tom M. 1997. Machine Learning. New York: McGraw-Hill. VanderPlas, Jake. 2017. Python Data Science Handbook. Sebastopol: O’Reilly.

184

|

Chapter 5: Machine Learning

CHAPTER 6

AI-First Finance

A computation takes information and transforms it, implementing what mathemati‐ cians call a function….If you’re in possession of a function that inputs all the world’s financial data and outputs the best stocks to buy, you’ll soon be extremely rich. —Max Tegmark (2017)

This chapter sets out to combine data-driven finance with the machine learning approach from the previous chapter. It only represents the beginning of this endeavor in that, for the first time, neural networks are used to discover statistical inefficien‐ cies. “Efficient Markets” on page 186 discusses the efficient market hypothesis and uses OLS regression to illustrate it based on financial time series data. “Market Pre‐ diction Based on Returns Data” on page 192 for the first time applies neural net‐ works, alongside OLS regression, to predict the future direction of a financial instrument’s price (“market direction”). The analysis relies on returns data only. “Market Prediction with More Features” on page 199 adds more features to the mix, such as typical financial indicators. In this context, first results indicate that statistical inefficiencies might indeed be present. This is confirmed in “Market Prediction Intra‐ day” on page 204, which works with intraday data as compared to end-of-day data. Finally, “Conclusions” on page 205 discusses the effectiveness of big data in combina‐ tion with AI in certain domains and argues that AI-first, theory-free finance might represent a way out of the theory fallacies in traditional finance.

185

Efficient Markets One of the hypotheses with the strongest empirical support is the efficient market hypothesis (EMH). It is also called the random walk hypothesis (RWH).1 Simply speak‐ ing, the hypothesis says that the prices of financial instruments at a certain point in time reflect all available information at this point in time. If the EMH holds true, a discussion about whether the price of a stock is too high or too low would be point‐ less. The price of a stock, given the EMH, is at all times exactly on its appropriate level given the available information. Lots of effort has been put into refining and formalizing the idea of efficient markets since the formulation and first discussions of the EMH in the 1960s. The definitions as presented in Jensen (1978) are still used today. Jensen defines an efficient market as follows: A market is efficient with respect to an information set θt if it is impossible to make economic profits by trading on the basis of information set θt. By economic profits, we mean the risk adjusted returns net of all costs.

In this context, Jensen distinguishes three forms of market efficiency: Weak form of EMH In this case, the information set θt only encompasses the past price and return history of the market. Semi-strong form of EMH In this case, the information set θt is taken to be all publicly available informa‐ tion, including not only the past price and return history but also financial reports, news articles, weather data, and so on. Strong form of EMH This case is given when the information set θt includes all information available to anyone (that is, even private information). No matter which form is assumed, the implications of the EMH are far reaching. In his pioneering article on the EMH, Fama (1965) concludes the following: For many years, economists, statisticians, and teachers of finance have been interested in developing and testing models of stock price behavior. One important model that has evolved from this research is the theory of random walks. This theory casts serious doubt on many other methods for describing and predicting stock price behavior— methods that have considerable popularity outside the academic world. For example, we shall see later that, if the random-walk theory is an accurate description of reality,

1 For the purposes of this chapter and the book, the two hypotheses are treated as equal, although the RWH is

somewhat stronger than the EMH. See, for instance, Copeland et al. (2005, ch. 10).

186

| Chapter 6: AI-First Finance

then the various “technical” or “chartist” procedures for predicting stock prices are completely without value.

In other words, if the EMH holds true, then any kind of research or data analysis for the purposes of achieving above-market returns should be useless in practice. On the other hand, a multitrillion-dollar asset management industry has evolved that prom‐ ises such above-market returns due to rigorous research and the active management of capital. In particular, the hedge fund industry is based on promises to deliver alpha —that is, returns that are above-market and even independent, at least to a large extent, of the market returns. How hard it is to live up to such a promise is shown by the data from a recent study by Preqin. The study reports a drop in the Preqin AllStrategies Hedge Fund index of –3.42% for the year 2018. Close to 40% of all hedge funds covered by the study experienced losses of 5% or greater for that year. If a stock price (or the price of any other financial instrument) follows a standard random walk, then the returns are normally distributed with zero mean. The stock price goes up with 50% probability and down with 50% probability. In such a context, the best predictor of tomorrow’s stock price, in a least-squares sense, is today’s stock price. This is due to the Markov property of random walks, namely that the distribu‐ tion of the future stock prices is independent of the history of the price process; it only depends on the current price level. Therefore, in the context of a random walk, the analysis of the historical prices (or returns) is useless for predicting future prices. Against this background, a semiformal test for efficient markets can be implemented as follows.2 Take a financial time series, lag the price data multiple times, and use the lagged price data as features data for an OLS regression that uses the current price level as the labels data. This is similar in spirit to charting techniques that rely on his‐ torical price formations to predict future prices. The following Python code implements such an analysis based on lagged price data for a number of financial instruments—both tradable ones and nontradable ones. First, import the data and its visualization (see Figure 6-1): In [1]: import numpy as np import pandas as pd from pylab import plt, mpl plt.style.use('seaborn') mpl.rcParams['savefig.dpi'] = 300 mpl.rcParams['font.family'] = 'serif' pd.set_option('precision', 4) np.set_printoptions(suppress=True, precision=4) In [2]: url = 'http://hilpisch.com/aiif_eikon_eod_data.csv' In [3]: data = pd.read_csv(url, index_col=0, parse_dates=True).dropna()

2 See also Hilpisch (2018, ch. 15).

Efficient Markets

|

187

In [4]: (data / data.iloc[0]).plot(figsize=(10, 6), cmap='coolwarm');

Reads the data into a DataFrame object Plots the normalized time series data

Figure 6-1. Normalized time series data (end-of-day) Second, the price data for all financial time series is lagged and stored in DataFrame objects: In [5]: lags = 7 In [6]: def add_lags(data, ric, lags): cols = [] df = pd.DataFrame(data[ric]) for lag in range(1, lags + 1): col = 'lag_{}'.format(lag) df[col] = df[ric].shift(lag) cols.append(col) df.dropna(inplace=True) return df, cols In [7]: dfs = {} for sym in data.columns: df, cols = add_lags(data, sym, lags) dfs[sym] = df In [8]: dfs[sym].head(7) Out[8]: GLD Date 2010-01-13 111.54

188

|

Chapter 6: AI-First Finance

lag_1

lag_2

lag_3

lag_4

110.49 112.85

111.37

110.82

lag_5

lag_6

lag_7

111.51 109.70

109.80

2010-01-14 2010-01-15 2010-01-19 2010-01-20 2010-01-21 2010-01-22

112.03 110.86 111.52 108.94 107.37 107.17

111.54 112.03 110.86 111.52 108.94 107.37

110.49 111.54 112.03 110.86 111.52 108.94

112.85 110.49 111.54 112.03 110.86 111.52

111.37 112.85 110.49 111.54 112.03 110.86

110.82 111.37 112.85 110.49 111.54 112.03

111.51 110.82 111.37 112.85 110.49 111.54

109.70 111.51 110.82 111.37 112.85 110.49

The number of lags (in trading days) Creates a column name Lags the price data Adds the column name to a list object Deletes all incomplete data rows Creates the lagged data for every financial time series Stores the results in a dict object Shows a sample of the lagged price data Third, with the data prepared, the OLS regression analysis is straightforward to con‐ duct. Figure 6-2 shows the average optimal regression results. Without a doubt, the price data that is lagged by only one day has the highest explanatory power. Its weight is close to 1, supporting the idea that the best predictor for tomorrow’s price of a financial instrument is its price today. This also holds true for the single regression results obtained per financial time series: In [9]: regs = {} for sym in data.columns: df = dfs[sym] reg = np.linalg.lstsq(df[cols], df[sym], rcond=-1)[0] regs[sym] = reg In [10]: rega = np.stack(tuple(regs.values())) In [11]: regd = pd.DataFrame(rega, columns=cols, index=data.columns) In [12]: regd Out[12]: AAPL.O MSFT.O INTC.O AMZN.O GS.N SPY .SPX

lag_1 lag_2 lag_3 lag_4 lag_5 lag_6 lag_7 1.0106 -0.0592 0.0258 0.0535 -0.0172 0.0060 -0.0184 0.8928 0.0112 0.1175 -0.0832 -0.0258 0.0567 0.0323 0.9519 0.0579 0.0490 -0.0772 -0.0373 0.0449 0.0112 0.9799 -0.0134 0.0206 0.0007 0.0525 -0.0452 0.0056 0.9806 0.0342 -0.0172 0.0042 -0.0387 0.0585 -0.0215 0.9692 0.0067 0.0228 -0.0244 -0.0237 0.0379 0.0121 0.9672 0.0106 0.0219 -0.0252 -0.0318 0.0515 0.0063

Efficient Markets

|

189

.VIX EUR= XAU= GDX GLD

0.8823 0.9859 0.9864 0.9765 0.9766

0.0591 -0.0289 0.0284 -0.0256 0.0239 -0.0484 0.0508 -0.0217 0.0069 0.0166 -0.0215 0.0044 0.0096 -0.0039 0.0223 -0.0364 0.0246 0.0060 -0.0142 -0.0047

0.0511 0.0149 0.0198 0.0379 0.0223

0.0306 -0.0055 -0.0125 -0.0065 -0.0106

In [13]: regd.mean().plot(kind='bar', figsize=(10, 6));

Gets the data for the current time series Implements the regression analysis Stores the optimal regression parameters in a dict object Combines the optimal results into a single ndarray object Puts the results into a DataFrame object and shows them Visualizes the average optimal regression parameters (weights) for every lag

Figure 6-2. Average optimal regression parameters for the lagged prices Given this semiformal analysis, there seems to be strong supporting evidence for the EMH in its weak form, at least. It is noteworthy that the OLS regression analysis as implemented here violates several assumptions. Among those is that the features are assumed to be noncorrelated among each other, whereas they should ideally be highly correlated with the labels data. However, the lagged price data leads to highly correla‐ ted features. The following Python code presents the correlation data, which shows a close-to-perfect correlation between all features. This explains why only one feature

190

|

Chapter 6: AI-First Finance

(“lag 1”) is enough to accomplish the approximation and prediction based on the OLS regression approach. Adding more, highly correlated features does not yield any improvements. Another fundamental assumption violated is the stationarity of the time series data, which the following code also tests for:3 In [14]: dfs[sym].corr() Out[14]: GLD lag_1 GLD 1.0000 0.9972 lag_1 0.9972 1.0000 lag_2 0.9946 0.9972 lag_3 0.9920 0.9946 lag_4 0.9893 0.9920 lag_5 0.9867 0.9893 lag_6 0.9841 0.9867 lag_7 0.9815 0.9842

lag_2 0.9946 0.9972 1.0000 0.9972 0.9946 0.9920 0.9893 0.9867

lag_3 0.9920 0.9946 0.9972 1.0000 0.9972 0.9946 0.9920 0.9893

lag_4 0.9893 0.9920 0.9946 0.9972 1.0000 0.9972 0.9946 0.9920

lag_5 0.9867 0.9893 0.9920 0.9946 0.9972 1.0000 0.9972 0.9946

lag_6 0.9841 0.9867 0.9893 0.9920 0.9946 0.9972 1.0000 0.9972

lag_7 0.9815 0.9842 0.9867 0.9893 0.9920 0.9946 0.9972 1.0000

In [15]: from statsmodels.tsa.stattools import adfuller In [16]: adfuller(data[sym].dropna()) Out[16]: (-1.9488969577009954, 0.3094193074034718, 0, 2515, {'1%': -3.4329527780962255, '5%': -2.8626898965523724, '10%': -2.567382133955709}, 8446.683102944744)

Shows the correlations between the lagged time series Tests for stationarity using the Augmented Dickey-Fuller test In summary, if the EMH holds true, active or algorithmic portfolio management or trading would not make economic sense. Simply investing in a stock or an efficient portfolio in the MVP sense, say, and passively holding the investment over a long period would yield without any effort at least the same, if not superior, returns. According to the CAPM and the MVP, the higher the risk the investor is willing to bear, the higher the expected return should be. In fact, as Copeland et al. (2005, ch. 10) point out, the CAPM and the EMH form a joint hypothesis about financial markets: if the EMH is rejected, then the CAPM must be rejected as well, since its derivation assumes the EMH to hold true.

3 For details on stationarity in financial time series, see Tsay (2005, sec. 2.1). Tsay points out: “The foundation of

time series analysis is stationarity.”

Efficient Markets

|

191

Market Prediction Based on Returns Data As Chapter 2 shows, ML and, in particular, DL algorithms have generated break‐ throughs in recent years in fields that have proven resistant over pretty long periods of time to standard statistical or mathematical methods. What about the financial markets? Might ML and DL algorithms be capable of discovering inefficiencies where traditional financial econometrics methods, such as OLS regression, fail? Of course, there are no simple and concise answers to these questions yet. However, some concrete examples might shed light on possible answers. To this end, the same data as in the previous section is used to derive log returns from the price data. The idea is to compare the performance of OLS regression to the performance of neural networks in predicting the next day’s direction of movement for the differ‐ ent time series. The goal at this stage is to discover statistical inefficiencies as com‐ pared to economic inefficiencies. Statistical inefficiencies are given when a model is able to predict the direction of the future price movement with a certain edge (say, the prediction is correct in 55% or 60% of the cases). Economic inefficiencies would only be given if the statistical inefficiencies can be exploited profitably through a trading strategy that takes into account, for example, transaction costs. The first step in the analysis is to create data sets with lagged log returns data. The normalized lagged log returns data is also tested for stationarity (given), and the fea‐ tures are tested for correlation (not correlated). Since the following analyses rely on time-series-related data only, they are dealing with weak form market efficiency: In [17]: rets = np.log(data / data.shift(1)) In [18]: rets.dropna(inplace=True) In [19]: dfs = {} for sym in data: df, cols = add_lags(rets, sym, lags) mu, std = df[cols].mean(), df[cols].std() df[cols] = (df[cols] - mu) / std dfs[sym] = df In [20]: dfs[sym].head() Out[20]: GLD Date 2010-01-14 0.0044 2010-01-15 -0.0105 2010-01-19 0.0059 2010-01-20 -0.0234 2010-01-21 -0.0145

lag_1 0.9570 0.4379 -1.0842 0.5967 -2.4045

lag_2

|

Chapter 6: AI-First Finance

lag_4

lag_5

lag_6

lag_7

-2.1692 1.3386 0.4959 -0.6434 1.6613 -0.1028 0.9571 -2.1689 1.3388 0.4966 -0.6436 1.6614 0.4385 0.9562 -2.1690 1.3395 0.4958 -0.6435 -1.0823 0.4378 0.9564 -2.1686 1.3383 0.4958 0.5971 -1.0825 0.4379 0.9571 -2.1680 1.3384

In [21]: adfuller(dfs[sym]['lag_1']) Out[21]: (-51.568251505825536, 0.0,

192

lag_3

0, 2507, {'1%': -3.4329610922579095, '5%': -2.8626935681060375, '10%': -2.567384088736619}, 7017.165474260225) In [22]: dfs[sym].corr() Out[22]: GLD lag_1 GLD 1.0000 -0.0297 lag_1 -0.0297 1.0000 lag_2 0.0003 -0.0305 lag_3 0.0126 0.0008 lag_4 -0.0026 0.0128 lag_5 -0.0059 -0.0029 lag_6 0.0099 -0.0053 lag_7 -0.0013 0.0098

lag_2 0.0003 -0.0305 1.0000 -0.0316 0.0003 0.0132 -0.0043 -0.0052

lag_3 lag_4 lag_5 lag_6 lag_7 1.2635e-02 -0.0026 -5.9392e-03 0.0099 -0.0013 8.1418e-04 0.0128 -2.8765e-03 -0.0053 0.0098 -3.1617e-02 0.0003 1.3234e-02 -0.0043 -0.0052 1.0000e+00 -0.0313 -6.8542e-06 0.0141 -0.0044 -3.1329e-02 1.0000 -3.1761e-02 0.0002 0.0141 -6.8542e-06 -0.0318 1.0000e+00 -0.0323 0.0002 1.4115e-02 0.0002 -3.2289e-02 1.0000 -0.0324 -4.3869e-03 0.0141 2.1707e-04 -0.0324 1.0000

Derives the log returns from the price data Lags the log returns data Applies Gaussian normalization to the features data4 Shows a sample of the lagged returns data Tests for stationarity of the time series data Shows the correlation data for the features First, the OLS regression is implemented and the predictions resulting from the regression are generated. The analysis is implemented on the complete data set. It shall show how well the algorithms perform in-sample. The accuracy with which OLS regression predicts the next day’s direction of movement is slightly, or even a few per‐ centage points, above 50% with one exception: In [23]: from sklearn.metrics import accuracy_score In [24]: %%time for sym in data: df = dfs[sym] reg = np.linalg.lstsq(df[cols], df[sym], rcond=-1)[0] pred = np.dot(df[cols], reg) acc = accuracy_score(np.sign(df[sym]), np.sign(pred)) print(f'OLS | {sym:10s} | acc={acc:.4f}') OLS | AAPL.O | acc=0.5056

4 Another term for the approach is z-score normalization.

Market Prediction Based on Returns Data

|

193

OLS | MSFT.O OLS | INTC.O OLS | AMZN.O OLS | GS.N OLS | SPY OLS | .SPX OLS | .VIX OLS | EUR= OLS | XAU= OLS | GDX OLS | GLD CPU times: user Wall time: 60.8

| acc=0.5088 | acc=0.5040 | acc=0.5048 | acc=0.5080 | acc=0.5080 | acc=0.5167 | acc=0.5291 | acc=0.4984 | acc=0.5207 | acc=0.5307 | acc=0.5072 201 ms, sys: 65.8 ms, total: 267 ms ms

The regression step The prediction step The accuracy of the prediction Second, the same analysis is done again but this time with a neural network from scikit-learn as the model for learning and predicting. The prediction accuracy insample is significantly above 50% throughout and above 60% in a few cases: In [25]: from sklearn.neural_network import MLPRegressor In [26]: %%time for sym in data.columns: df = dfs[sym] model = MLPRegressor(hidden_layer_sizes=[512], random_state=100, max_iter=1000, early_stopping=True, validation_fraction=0.15, shuffle=False) model.fit(df[cols], df[sym]) pred = model.predict(df[cols]) acc = accuracy_score(np.sign(df[sym]), np.sign(pred)) print(f'MLP | {sym:10s} | acc={acc:.4f}') MLP | AAPL.O | acc=0.6005 MLP | MSFT.O | acc=0.5853 MLP | INTC.O | acc=0.5766 MLP | AMZN.O | acc=0.5510 MLP | GS.N | acc=0.6527 MLP | SPY | acc=0.5419 MLP | .SPX | acc=0.5399 MLP | .VIX | acc=0.6579 MLP | EUR= | acc=0.5642 MLP | XAU= | acc=0.5522 MLP | GDX | acc=0.6029 MLP | GLD | acc=0.5259

194

|

Chapter 6: AI-First Finance

CPU times: user 1min 37s, sys: 6.74 s, total: 1min 44s Wall time: 14 s

Model instantiation Model fitting Prediction step Accuracy calculation Third, the same analysis again but with a neural network from the Keras package. The accuracy results are similar to those from the MLPRegressor, but with a higher average accuracy: In [27]: import tensorflow as tf from keras.layers import Dense from keras.models import Sequential Using TensorFlow backend. In [28]: np.random.seed(100) tf.random.set_seed(100) In [29]: def create_model(problem='regression'): model = Sequential() model.add(Dense(512, input_dim=len(cols), activation='relu')) if problem == 'regression': model.add(Dense(1, activation='linear')) model.compile(loss='mse', optimizer='adam') else: model.add(Dense(1, activation='sigmoid')) model.compile(loss='binary_crossentropy', optimizer='adam') return model In [30]: %%time for sym in data.columns[:]: df = dfs[sym] model = create_model() model.fit(df[cols], df[sym], epochs=25, verbose=False) pred = model.predict(df[cols]) acc = accuracy_score(np.sign(df[sym]), np.sign(pred)) print(f'DNN | {sym:10s} | acc={acc:.4f}') DNN | AAPL.O | acc=0.6292 DNN | MSFT.O | acc=0.5981 DNN | INTC.O | acc=0.6073 DNN | AMZN.O | acc=0.5781 DNN | GS.N | acc=0.6196 DNN | SPY | acc=0.5829 DNN | .SPX | acc=0.6077 DNN | .VIX | acc=0.6392

Market Prediction Based on Returns Data

|

195

DNN | EUR= | acc=0.5845 DNN | XAU= | acc=0.5881 DNN | GDX | acc=0.5829 DNN | GLD | acc=0.5666 CPU times: user 34.3 s, sys: 5.34 s, total: 39.6 s Wall time: 23.1 s

Model creation function Model instantiation Model fitting Prediction step Accuracy calculation This simple example shows that neural networks can outperform OLS regression sig‐ nificantly in-sample in predicting the next day’s direction of price movements. How‐ ever, how does the picture change when testing for the out-of-sample performance of the two model types? To this end, the analyses are repeated, but the training (fitting) step is implemented on the first 80% of the data while the performance is tested on the remaining 20%. OLS regression is implemented first. Out-of-sample OLS regression shows similar accuracy levels as in-sample—around 50%: In [31]: split = int(len(dfs[sym]) * 0.8) In [32]: %%time for sym in data.columns: df = dfs[sym] train = df.iloc[:split] reg = np.linalg.lstsq(train[cols], train[sym], rcond=-1)[0] test = df.iloc[split:] pred = np.dot(test[cols], reg) acc = accuracy_score(np.sign(test[sym]), np.sign(pred)) print(f'OLS | {sym:10s} | acc={acc:.4f}') OLS | AAPL.O | acc=0.5219 OLS | MSFT.O | acc=0.4960 OLS | INTC.O | acc=0.5418 OLS | AMZN.O | acc=0.4841 OLS | GS.N | acc=0.4980 OLS | SPY | acc=0.5020 OLS | .SPX | acc=0.5120 OLS | .VIX | acc=0.5458 OLS | EUR= | acc=0.4482 OLS | XAU= | acc=0.5299 OLS | GDX | acc=0.5159 OLS | GLD | acc=0.5100

196

|

Chapter 6: AI-First Finance

CPU times: user 200 ms, sys: 60.6 ms, total: 261 ms Wall time: 61.7 ms

Creates the training data sub-set Creates the test data sub-set The performance of the MLPRegressor model is out-of-sample much worse when compared to the in-sample numbers and similar to the OLS regression results: In [34]: %%time for sym in data.columns: df = dfs[sym] train = df.iloc[:split] model = MLPRegressor(hidden_layer_sizes=[512], random_state=100, max_iter=1000, early_stopping=True, validation_fraction=0.15, shuffle=False) model.fit(train[cols], train[sym]) test = df.iloc[split:] pred = model.predict(test[cols]) acc = accuracy_score(np.sign(test[sym]), np.sign(pred)) print(f'MLP | {sym:10s} | acc={acc:.4f}') MLP | AAPL.O | acc=0.4920 MLP | MSFT.O | acc=0.5279 MLP | INTC.O | acc=0.5279 MLP | AMZN.O | acc=0.4641 MLP | GS.N | acc=0.5040 MLP | SPY | acc=0.5259 MLP | .SPX | acc=0.5478 MLP | .VIX | acc=0.5279 MLP | EUR= | acc=0.4980 MLP | XAU= | acc=0.5239 MLP | GDX | acc=0.4880 MLP | GLD | acc=0.5000 CPU times: user 1min 39s, sys: 4.98 s, total: 1min 44s Wall time: 13.7 s

The same holds true for the Sequential model from Keras for which the out-ofsample numbers also show accuracy values between a few percentage points above and below the 50% threshold: In [35]: %%time for sym in data.columns: df = dfs[sym] train = df.iloc[:split] model = create_model() model.fit(train[cols], train[sym], epochs=50, verbose=False) test = df.iloc[split:] pred = model.predict(test[cols])

Market Prediction Based on Returns Data

|

197

acc = accuracy_score(np.sign(test[sym]), np.sign(pred)) print(f'DNN | {sym:10s} | acc={acc:.4f}') DNN | AAPL.O | acc=0.5179 DNN | MSFT.O | acc=0.5598 DNN | INTC.O | acc=0.4821 DNN | AMZN.O | acc=0.4920 DNN | GS.N | acc=0.5179 DNN | SPY | acc=0.4861 DNN | .SPX | acc=0.5100 DNN | .VIX | acc=0.5378 DNN | EUR= | acc=0.4661 DNN | XAU= | acc=0.4602 DNN | GDX | acc=0.4841 DNN | GLD | acc=0.5378 CPU times: user 50.4 s, sys: 7.52 s, total: 57.9 s Wall time: 32.9 s

Weak Form Market Efficiency Although the labeling as weak form market efficiency might suggest otherwise, it is the hardest form in the sense that only time-seriesrelated data can be used to identify statistical inefficiencies. With the semi-strong form, any other source of publicly available data could be added to improve prediction accuracy.

Based on the approaches chosen in this section, markets seem to be at least efficient in the weak form. Just analyzing historical return patterns based on OLS regression or neural networks might not be enough to discover statistical inefficiencies. There are two major elements of the approach chosen in this section that can be adjusted in the hope of improving prediction results: Features In addition to vanilla price-and-returns data, other features can be added to the data, such as technical indicators (for example, simple moving averages, or SMAs for short). The hope is, in the technical chartist’s tradition, that such indicators improve the prediction accuracy. Bar length Instead of working with end-of-day data, intraday data might allow for higher prediction accuracies. Here, the hope is that one is more likely to discover statis‐ tical inefficiencies during the day as compared to at end of day, when all market participants in general pay the highest attention to making their final trades—by taking into account all available information. The following two sections address these elements.

198

|

Chapter 6: AI-First Finance

Market Prediction with More Features In trading, there is a long tradition of using technical indicators to generate, based on observed patterns, buy or sell signals. Such technical indicators, basically of any kind, can also be used as features for the training of neural networks. The following Python code uses an SMA, rolling minimum and maximum values, momentum, and rolling volatility as features: In [36]: url = 'http://hilpisch.com/aiif_eikon_eod_data.csv' In [37]: data = pd.read_csv(url, index_col=0, parse_dates=True).dropna() In [38]: def add_lags(data, ric, lags, window=50): cols = [] df = pd.DataFrame(data[ric]) df.dropna(inplace=True) df['r'] = np.log(df / df.shift()) df['sma'] = df[ric].rolling(window).mean() df['min'] = df[ric].rolling(window).min() df['max'] = df[ric].rolling(window).max() df['mom'] = df['r'].rolling(window).mean() df['vol'] = df['r'].rolling(window).std() df.dropna(inplace=True) df['d'] = np.where(df['r'] > 0, 1, 0) features = [ric, 'r', 'd', 'sma', 'min', 'max', 'mom', 'vol'] for f in features: for lag in range(1, lags + 1): col = f'{f}_lag_{lag}' df[col] = df[f].shift(lag) cols.append(col) df.dropna(inplace=True) return df, cols In [39]: lags = 5 In [40]: dfs = {} for ric in data: df, cols = add_lags(data, ric, lags) dfs[ric] = df.dropna(), cols

Simple moving average (SMA) Rolling minimum Rolling maximum Momentum as average of log returns Rolling volatility Market Prediction with More Features

|

199

Direction as binary feature

Technical Indicators as Features As the preceding examples show, basically any traditional technical indicator used for investing or intraday trading can be used as a feature to train ML algorithms. In that sense, AI and ML do not necessarily render such indicators obsolete, rather they can indeed enrich the ML-driven derivation of trading strategies.

In-sample, the performance of the MLPClassifier model is now much better when taking into account the new features and normalizing them for training. The Sequential model of Keras reaches accuracies of around 70% for the number of epochs trained. From experience, these can be easily increased by increasing the number of epochs and/or the capacity of the neural network: In [41]: from sklearn.neural_network import MLPClassifier In [42]: %%time for ric in data: model = MLPClassifier(hidden_layer_sizes=[512], random_state=100, max_iter=1000, early_stopping=True, validation_fraction=0.15, shuffle=False) df, cols = dfs[ric] df[cols] = (df[cols] - df[cols].mean()) / df[cols].std() model.fit(df[cols], df['d']) pred = model.predict(df[cols]) acc = accuracy_score(df['d'], pred) print(f'IN-SAMPLE | {ric:7s} | acc={acc:.4f}') IN-SAMPLE | AAPL.O | acc=0.5510 IN-SAMPLE | MSFT.O | acc=0.5376 IN-SAMPLE | INTC.O | acc=0.5607 IN-SAMPLE | AMZN.O | acc=0.5559 IN-SAMPLE | GS.N | acc=0.5794 IN-SAMPLE | SPY | acc=0.5729 IN-SAMPLE | .SPX | acc=0.5941 IN-SAMPLE | .VIX | acc=0.6940 IN-SAMPLE | EUR= | acc=0.5766 IN-SAMPLE | XAU= | acc=0.5672 IN-SAMPLE | GDX | acc=0.5847 IN-SAMPLE | GLD | acc=0.5567 CPU times: user 1min 1s, sys: 4.5 s, total: 1min 6s Wall time: 9.05 s In [43]: %%time for ric in data:

200

| Chapter 6: AI-First Finance

model = create_model('classification') df, cols = dfs[ric] df[cols] = (df[cols] - df[cols].mean()) / df[cols].std() model.fit(df[cols], df['d'], epochs=50, verbose=False) pred = np.where(model.predict(df[cols]) > 0.5, 1, 0) acc = accuracy_score(df['d'], pred) print(f'IN-SAMPLE | {ric:7s} | acc={acc:.4f}') IN-SAMPLE | AAPL.O | acc=0.7156 IN-SAMPLE | MSFT.O | acc=0.7156 IN-SAMPLE | INTC.O | acc=0.7046 IN-SAMPLE | AMZN.O | acc=0.6640 IN-SAMPLE | GS.N | acc=0.6855 IN-SAMPLE | SPY | acc=0.6696 IN-SAMPLE | .SPX | acc=0.6579 IN-SAMPLE | .VIX | acc=0.7489 IN-SAMPLE | EUR= | acc=0.6737 IN-SAMPLE | XAU= | acc=0.7143 IN-SAMPLE | GDX | acc=0.6826 IN-SAMPLE | GLD | acc=0.7078 CPU times: user 1min 5s, sys: 7.06 s, total: 1min 12s Wall time: 44.3 s

Normalizes the features data Are these improvements to be transferred to the out-of-sample prediction accuracies? The following Python code repeats the analysis, this time with the training and test split as used before. Unfortunately, the picture is mixed at best. The numbers do not represent real improvements when compared to the approach, relying only on lagged returns data as features. For selected instruments, there seems to be an edge of a few percentage points in the prediction accuracy compared to the 50% benchmark. For others, however, the accuracy is still below 50%—as illustrated for the MLPClassifier model: In [44]: def train_test_model(model): for ric in data: df, cols = dfs[ric] split = int(len(df) * 0.85) train = df.iloc[:split].copy() mu, std = train[cols].mean(), train[cols].std() train[cols] = (train[cols] - mu) / std model.fit(train[cols], train['d']) test = df.iloc[split:].copy() test[cols] = (test[cols] - mu) / std pred = model.predict(test[cols]) acc = accuracy_score(test['d'], pred) print(f'OUT-OF-SAMPLE | {ric:7s} | acc={acc:.4f}') In [45]: model_mlp = MLPClassifier(hidden_layer_sizes=[512], random_state=100, max_iter=1000, early_stopping=True,

Market Prediction with More Features

|

201

validation_fraction=0.15, shuffle=False) In [46]: %time train_test_model(model_mlp) OUT-OF-SAMPLE | AAPL.O | acc=0.4432 OUT-OF-SAMPLE | MSFT.O | acc=0.4595 OUT-OF-SAMPLE | INTC.O | acc=0.5000 OUT-OF-SAMPLE | AMZN.O | acc=0.5270 OUT-OF-SAMPLE | GS.N | acc=0.4838 OUT-OF-SAMPLE | SPY | acc=0.4811 OUT-OF-SAMPLE | .SPX | acc=0.5027 OUT-OF-SAMPLE | .VIX | acc=0.5676 OUT-OF-SAMPLE | EUR= | acc=0.4649 OUT-OF-SAMPLE | XAU= | acc=0.5514 OUT-OF-SAMPLE | GDX | acc=0.5162 OUT-OF-SAMPLE | GLD | acc=0.4946 CPU times: user 44.9 s, sys: 2.64 s, total: 47.5 s Wall time: 6.37 s

Training data set statistics are used for normalization. The good in-sample performance and the not-so-good out-of-sample performance suggest that overfitting of the neural network might play a crucial role. One approach to avoid overfitting is to use ensemble methods that combine multiple trained models of the same type to come up with a more robust meta model and better out-of-sample predictions. One such method is called bagging. scikit-learn has an implementa‐ tion of this approach in the form of the BaggingClassifier class. Using multiple estimators allows for training every one of them without exposing them to the com‐ plete training data set or all features. This should help in avoiding overfitting. The following Python code implements a bagging approach based on a number of base estimators of the same type (MLPClassifier). The prediction accuracies are now consistently above 50%. Some accuracy values are above 55%, which can be consid‐ ered pretty high in this context. Overall, bagging seems to avoid, at least to some extent, overfitting and seems to improve the predictions noticeably: In [47]: from sklearn.ensemble import BaggingClassifier In [48]: base_estimator = MLPClassifier(hidden_layer_sizes=[256], random_state=100, max_iter=1000, early_stopping=True, validation_fraction=0.15, shuffle=False) In [49]: model_bag = BaggingClassifier(base_estimator=base_estimator, n_estimators=35, max_samples=0.25, max_features=0.5, bootstrap=False,

202

|

Chapter 6: AI-First Finance

bootstrap_features=True, n_jobs=8, random_state=100 ) In [50]: %time train_test_model(model_bag) OUT-OF-SAMPLE | AAPL.O | acc=0.5243 OUT-OF-SAMPLE | MSFT.O | acc=0.5703 OUT-OF-SAMPLE | INTC.O | acc=0.5027 OUT-OF-SAMPLE | AMZN.O | acc=0.5270 OUT-OF-SAMPLE | GS.N | acc=0.5243 OUT-OF-SAMPLE | SPY | acc=0.5595 OUT-OF-SAMPLE | .SPX | acc=0.5514 OUT-OF-SAMPLE | .VIX | acc=0.5649 OUT-OF-SAMPLE | EUR= | acc=0.5108 OUT-OF-SAMPLE | XAU= | acc=0.5378 OUT-OF-SAMPLE | GDX | acc=0.5162 OUT-OF-SAMPLE | GLD | acc=0.5432 CPU times: user 2.55 s, sys: 494 ms, total: 3.05 s Wall time: 11.1 s

The base estimator The number of estimators used Maximum percentage of training data used per estimator Maximum number of features used per estimator Whether to bootstrap (reuse) data Whether to bootstrap (reuse) features Number of parallel jobs

End-of-Day Market Efficiency The efficient market hypothesis dates back to the 1960s and 1970s, periods during which end-of-day data was basically the only avail‐ able time series data. Back in those days (and still today), it could be assumed that market players paid particularly close attention to their positions and trades the closer the end of the trading session came. This might be more true for stocks, say, and a bit less so for currencies, which are traded in principle around the clock.

Market Prediction with More Features

|

203

Market Prediction Intraday This chapter has not produced conclusive evidence, but the analyses implemented so far point more in the direction that markets are weakly efficient on an end-of-day basis. What about intraday markets? Are there more consistent statistical inefficien‐ cies to be spotted? To work toward an answer of this question, another data set is nec‐ essary. The following Python code uses a data set that is composed of the same instruments as in the end-of-day data set, but now contains hourly closing prices. Since trading hours might differ from instrument to instrument, the data set is incomplete. This is no problem, though, since the analyses are implemented time series by time series. The technical implementation for the hourly data is essentially the same as before, relying on the same code as the end-of-day analysis: In [51]: url = 'http://hilpisch.com/aiif_eikon_id_data.csv' In [52]: data = pd.read_csv(url, index_col=0, parse_dates=True) In [53]: data.info()

DatetimeIndex: 5529 entries, 2019-03-01 00:00:00 to 2020-01-01 00:00:00 Data columns (total 12 columns): # Column Non-Null Count Dtype --- ------ -------------- ----0 AAPL.O 3384 non-null float64 1 MSFT.O 3378 non-null float64 2 INTC.O 3275 non-null float64 3 AMZN.O 3381 non-null float64 4 GS.N 1686 non-null float64 5 SPY 3388 non-null float64 6 .SPX 1802 non-null float64 7 .VIX 2959 non-null float64 8 EUR= 5429 non-null float64 9 XAU= 5149 non-null float64 10 GDX 3173 non-null float64 11 GLD 3351 non-null float64 dtypes: float64(12) memory usage: 561.5 KB In [54]: lags = 5 In [55]: dfs = {} for ric in data: df, cols = add_lags(data, ric, lags) dfs[ric] = df, cols

The prediction accuracies intraday are again distributed around 50% with a relatively wide spread for the single neural network. On the positive side, some accuracy values are above 55%. The bagging meta model shows a more consistent out-of-sample 204

|

Chapter 6: AI-First Finance

performance, though, with many of the observed accuracy values a few percentage points above the 50% benchmark: In [56]: %time train_test_model(model_mlp) OUT-OF-SAMPLE | AAPL.O | acc=0.5420 OUT-OF-SAMPLE | MSFT.O | acc=0.4930 OUT-OF-SAMPLE | INTC.O | acc=0.5549 OUT-OF-SAMPLE | AMZN.O | acc=0.4709 OUT-OF-SAMPLE | GS.N | acc=0.5184 OUT-OF-SAMPLE | SPY | acc=0.4860 OUT-OF-SAMPLE | .SPX | acc=0.5019 OUT-OF-SAMPLE | .VIX | acc=0.4885 OUT-OF-SAMPLE | EUR= | acc=0.5130 OUT-OF-SAMPLE | XAU= | acc=0.4824 OUT-OF-SAMPLE | GDX | acc=0.4765 OUT-OF-SAMPLE | GLD | acc=0.5455 CPU times: user 1min 4s, sys: 5.05 s, total: 1min 9s Wall time: 9.56 s In [57]: %time train_test_model(model_bag) OUT-OF-SAMPLE | AAPL.O | acc=0.5660 OUT-OF-SAMPLE | MSFT.O | acc=0.5431 OUT-OF-SAMPLE | INTC.O | acc=0.5072 OUT-OF-SAMPLE | AMZN.O | acc=0.5110 OUT-OF-SAMPLE | GS.N | acc=0.5020 OUT-OF-SAMPLE | SPY | acc=0.5120 OUT-OF-SAMPLE | .SPX | acc=0.4677 OUT-OF-SAMPLE | .VIX | acc=0.5092 OUT-OF-SAMPLE | EUR= | acc=0.5242 OUT-OF-SAMPLE | XAU= | acc=0.5255 OUT-OF-SAMPLE | GDX | acc=0.5085 OUT-OF-SAMPLE | GLD | acc=0.5374 CPU times: user 2.64 s, sys: 439 ms, total: 3.08 s Wall time: 12.4 s

Intraday Market Efficiency Even if markets are weakly efficient on an end-of-day basis, they can nevertheless be weakly inefficient intraday. Such statistical ineffi‐ ciencies might result from temporary imbalances, buy or sell pres‐ sures, market overreactions, technically driven buy or sell orders, and so on. The central question is whether such statistical ineffi‐ ciencies, once discovered, can be exploited profitably via specific trading strategies.

Conclusions In their widely cited article “The Unreasonable Effectiveness of Data,” Halevy et al. (2009) point out that economists suffer from what they call physics envy. By that, they mean the inability to explain human behavior in the same mathematically elegant Conclusions

|

205

way that physicists are able to describe even complex real-world phenomena. One such example is Albert Einstein’s probably best-known formula E = mc2, which equa‐ tes energy with the mass of an object times the speed of light squared. In economics and finance, researchers for decades have tried to emulate the physical approach in deriving and proving simple, elegant equations to explain economic and financial phenomena. But as Chapter 3 and Chapter 4 together show, many of the most elegant financial theories have hardly any supporting evidence in the real finan‐ cial world in which the simplifying assumptions, such as normal distributions and linear relationships, do not hold. As Halevy et al. (2009) explain in their article, there might be domains, such as natu‐ ral languages and the rules they follow, that defy the derivation and formulation of concise, elegant theories. Researchers might simply need to rely on complex theories and models that are driven by data. For language in particular, the World Wide Web represents a treasure trove of big data. And big data seems to be required to train ML and DL algorithms on certain tasks, such as natural language processing or transla‐ tion on a human level. After all, finance might be a discipline that has more in common with natural lan‐ guage than with physics. Maybe there are, after all, no simple, elegant formulas that describe important financial phenomena, such as the daily change in a currency rate or the price of a stock.5 Maybe the truth might be found only in the big data that nowadays is available in programmatic fashion to financial researchers and academics alike. This chapter presents the beginning of the quest to uncover the truth, to discover the holy grail of finance: proving that markets are not that efficient after all. The relatively simple neural network approaches of this chapter only rely on time-series-related fea‐ tures for the training. The labels are simple and straightforward: whether the market (financial instrument’s price) goes up or down. The goal is to discover statistical inef‐ ficiencies in predicting the future market direction. This in turn represents the first step in exploiting such inefficiencies economically through an implementable trading strategy. Agrawal et al. (2018) explain in detail, with many examples, that predictions them‐ selves are only one side of the coin. Decision and implementation rules that specify in detail how a certain prediction is dealt with are equally important. The same holds true in an algorithmic trading context: the signal (prediction) is only the beginning.

5 There are, of course, more simple financial aspects that allow the modeling by a simple formula. An example

might be the derivation of a continuous discount factor D for a period of two years T = 2 if the relevant log return is r = 0.01. It is given by D r, T = exp − rT = exp -0.01 · 2 = 0.9802. AI or ML cannot offer any benefits here.

206

|

Chapter 6: AI-First Finance

The hard part is to optimally execute an appropriate trade, to monitor active trades, to implement appropriate risk measures—such as stop loss and take profit orders— and so on. In its quest for statistical inefficiencies, this chapter relies on data and neural net‐ works only. There is no theory involved, and there are no assumptions about how market participants might behave, or similar reasonings. The major modeling effort is done with regard to preparing the features, which of course represent what the modeler considers important. One implicit assumption in the approach taken is that statistical inefficiencies can be discovered based on time-series-related data only. This is to say that markets are not even weakly efficient—the most difficult form of the three to disprove. Relying on financial data only and applying general ML and DL algorithms and mod‐ els to it are what this book considers AI-first finance. No theories needed, no model‐ ing of human behavior, no assumptions about distributions or the nature of relationships—just data and algorithms. In that sense, AI-first finance could also be labeled theory-free or model-free finance.

References Books and papers cited in this chapter: Agrawal, Ajay, Joshua Gans, and Avi Goldfarb. 2018. Prediction Machines: The Simple Economics of Artificial Intelligence. Boston: Harvard Business Review Press. Copeland, Thomas, Fred Weston, and Kuldeep Shastri. 2005. Financial Theory and Corporate Policy. 4th ed. Boston: Pearson. Fama, Eugene. 1965. “Random Walks in Stock Market Prices.” Financial Analysts Journal (September/October): 55-59. Halevy, Alon, Peter Norvig, and Fernando Pereira. 2009. “The Unreasonable Effec‐ tiveness of Data.” IEEE Intelligent Systems, Expert Opinion. Hilpisch, Yves. 2018. Python for Finance: Mastering Data-Driven Finance. 2nd ed. Sebastopol: O’Reilly. Jensen, Michael. 1978. “Some Anomalous Evidence Regarding Market Efficiency.” Journal of Financial Economics 6 (2/3): 95-101. Tegmark, Max. 2017. Life 3.0: Being Human in the Age of Artificial Intelligence. United Kingdom: Penguin Random House. Tsay, Ruey S. 2005. Analysis of Financial Time Series. Hoboken: Wiley.

References

|

207

PART III

Statistical Inefficiencies

“There are patterns in the market,” Simons told a colleague. “I know we can find them.”1 —Gregory Zuckerman (2019)

The major goal of this part is to apply neural networks and reinforcement learning to discover statistical inefficiencies in financial markets (data). A statistical inefficiency, for the purposes of this book, is found when a predictor (a model or algorithm in gen‐ eral or a neural network in particular) predicts markets significantly better than a random predictor assigning equal probability to upwards and downwards move‐ ments. In an algorithmic trading context, to have such a predictor available is a pre‐ requisite for the generation of alpha or above-market returns. This part consists of three chapters that provide more background, details, and exam‐ ples related to dense neural networks (DNNs), recurrent neural networks (RNNs), and reinforcement learning (RL): • Chapter 7 covers DNNs in some more detail and applies them to the problem of predicting the direction of financial market movements. Historical data is used to generate lagged features data and to generate binary labels data. Such data sets are then used to train DNNs via supervised learning. The focus lies on identify‐ ing statistical inefficiencies in financial markets. In some of the examples, the DNN achieves an out-of-sample prediction accuracy of more than 60%.

1 Gregory Zuckerman. 2019. The Man Who Solved the Market. New York: Penguin Random House.

• Chapter 8 is about RNNs, which are designed to accommodate the specific nature of sequential data, such as textual data or time series data. The idea is to add some form of memory to the network that carries previous (historical) infor‐ mation through the network (layers). The approach taken in this chapter is close to the one in Chapter 7, with the same goal of discovering statistical inefficiencies in the financial market data. As numerical examples illustrate, RNNs also can reach prediction accuracies out-of-sample of more than 60%. • Chapter 9 discusses RL as one of the major success stories in AI. The chapter dis‐ cusses different RL agents applied to both a simulated physical environment from the OpenAI Gym and financial market environments as developed in the chapter. The algorithm of choice in RL often is Q-learning, which is discussed in detail and applied to train a trading bot. The trading bot shows respectable out-ofsample financial performance, which is generally an even more important yard‐ stick than prediction accuracy alone. In that sense, the chapter builds a natural bridge to Part IV, which is concerned with exploiting statistical inefficiencies economically. Although they are quite an important type of a neural network, convolutional neural networks (CNNs) are not discussed in detail in this part. Appendix C illustrates the application of CNNs in a concise way. In many cases, CNNs can also be applied to the problems that DNNs and RNNs are applied to in this part of the book. The approach in this part is a practical one, leaving out many important details with regard to the algorithms and techniques applied. This seems justified since there are a number of good resources in book form and otherwise available that can be consulted for technical details and background information. The chapters to follow provide ref‐ erences to a select few, generally comprehensive, resources when appropriate.

CHAPTER 7

Dense Neural Networks

[I]f you’re trying to predict the movements of a stock on the stock market given its recent price history, you’re unlikely to succeed, because price history doesn’t contain much predictive information. —François Chollet (2017)

This chapter is about important aspects of dense neural networks. Previous chapters have already made use of this type of neural network. In particular, the MLPClassifier and MLPRegressor models from scikit-learn and the Sequential model from Keras for classification and estimation are dense neural networks (DNNs). This chapter exclusively focuses on Keras since it gives more freedom and flexibility in modeling DNNs.1 “The Data” on page 212 introduces the foreign exchange (FX) data set that the other sections in this chapter use. “Baseline Prediction” on page 214 generates a baseline, in-sample prediction on the new data set. Normalization of training and test data is introduced in “Normalization” on page 218. As means to avoid overfitting, “Dropout” on page 220 and “Regularization” on page 222 discuss dropout and regularization as popular methods. Bagging, as another method to avoid overfitting and already used in Chapter 6, is revisited in “Bagging” on page 225. Finally, “Optimizers” on page 227 compares the performance of different optimizers that can be used with Keras DNN models. Although the introductory quote for the chapter might give little reason for hope, the main goal for this chapter—as well as for Part III as a whole—is to discover statistical inefficiencies in financial markets (time series) by applying neural networks. The

1 See Chollet (2017) for more details and background information on the Keras package. See Goodfellow et al.

(2016) for a comprehensive treatment of neural networks and related methods.

211

numerical results presented in this chapter, such as prediction accuracies of 60% and more in certain cases, indicate that at least some hope is justified.

The Data Chapter 6 discovers hints for statistical inefficiencies for, among other time series, the intraday price series of the EUR/USD currency pair. This chapter and the following ones focus on foreign exchange (FX) as an asset class and specifically on the EUR/USD currency pair. Among other reasons, economically exploiting statistical inefficiencies for FX is in general not as involved as for other asset classes, such as for volatility products like the VIX volatility index. Free and comprehensive data availa‐ bility is also often given for FX. The following data set is from the Refinitiv Eikon Data API. The data set has been retrieved via the API. The data set contains open, high, low, and close values. Figure 7-1 visualizes the closing prices: In [1]: import os import numpy as np import pandas as pd from pylab import plt, mpl plt.style.use('seaborn') mpl.rcParams['savefig.dpi'] = 300 mpl.rcParams['font.family'] = 'serif' pd.set_option('precision', 4) np.set_printoptions(suppress=True, precision=4) os.environ['PYTHONHASHSEED'] = '0' In [2]: url = 'http://hilpisch.com/aiif_eikon_id_eur_usd.csv' In [3]: symbol = 'EUR_USD' In [4]: raw = pd.read_csv(url, index_col=0, parse_dates=True) In [5]: raw.head() Out[5]: Date 2019-10-01 2019-10-01 2019-10-01 2019-10-01 2019-10-01

HIGH 00:00:00 00:01:00 00:02:00 00:03:00 00:04:00

1.0899 1.0899 1.0898 1.0898 1.0898

LOW

OPEN

CLOSE

1.0897 1.0897 1.0899 1.0896 1.0899 1.0898 1.0896 1.0898 1.0896 1.0896 1.0897 1.0898 1.0896 1.0897 1.0898

In [6]: raw.info()

DatetimeIndex: 96526 entries, 2019-10-01 00:00:00 to 2019-12-31 23:06:00 Data columns (total 4 columns): # Column Non-Null Count Dtype --- ------ -------------- ----0 HIGH 96526 non-null float64 1 LOW 96526 non-null float64

212

|

Chapter 7: Dense Neural Networks

2 OPEN 96526 non-null float64 3 CLOSE 96526 non-null float64 dtypes: float64(4) memory usage: 3.7 MB In [7]: data = pd.DataFrame(raw['CLOSE'].loc[:]) data.columns = [symbol] In [8]: data = data.resample('1h', label='right').last().ffill() In [9]: data.info()

DatetimeIndex: 2208 entries, 2019-10-01 01:00:00 to 2020-01-01 00:00:00 Freq: H Data columns (total 1 columns): # Column Non-Null Count Dtype --- ------------------- ----0 EUR_USD 2208 non-null float64 dtypes: float64(1) memory usage: 34.5 KB In [10]: data.plot(figsize=(10, 6));

Reads the data into a DataFrame object Selects, resamples, and plots the closing prices

Figure 7-1. Mid-closing prices for EUR/USD (intraday)

The Data

|

213

Baseline Prediction Based on the new data set, the prediction approach from Chapter 6 is repeated. First is the creation of the lagged features: In [11]: lags = 5 In [12]: def add_lags(data, symbol, lags, window=20): cols = [] df = data.copy() df.dropna(inplace=True) df['r'] = np.log(df / df.shift()) df['sma'] = df[symbol].rolling(window).mean() df['min'] = df[symbol].rolling(window).min() df['max'] = df[symbol].rolling(window).max() df['mom'] = df['r'].rolling(window).mean() df['vol'] = df['r'].rolling(window).std() df.dropna(inplace=True) df['d'] = np.where(df['r'] > 0, 1, 0) features = [symbol, 'r', 'd', 'sma', 'min', 'max', 'mom', 'vol'] for f in features: for lag in range(1, lags + 1): col = f'{f}_lag_{lag}' df[col] = df[f].shift(lag) cols.append(col) df.dropna(inplace=True) return df, cols In [13]: data, cols = add_lags(data, symbol, lags)

Slightly adjusted function from Chapter 6 Second, a look at the labels data. A major problem in classification that can arise depending on the data set available is class imbalance. This means, in the context of binary labels, that the frequency of one particular class compared to the other class might be higher. This might lead to situations in which the neural network simply predicts the class with the higher frequency since this already can lead to low loss and high accuracy values. Applying appropriate weights, one can make sure that both classes gain equal importance during the DNN training step:2 In [14]: len(data) Out[14]: 2183 In [15]: c = data['d'].value_counts() c Out[15]: 0 1445 1 738

2 See this blog post, which discusses solutions to class imbalance with Keras.

214

| Chapter 7: Dense Neural Networks

Name: d, dtype: int64 In [16]: def cw(df): c0, c1 = np.bincount(df['d']) w0 = (1 / c0) * (len(df)) / 2 w1 = (1 / c1) * (len(df)) / 2 return {0: w0, 1: w1} In [17]: class_weight = cw(data) In [18]: class_weight Out[18]: {0: 0.755363321799308, 1: 1.4789972899728998} In [19]: class_weight[0] * c[0] Out[19]: 1091.5 In [20]: class_weight[1] * c[1] Out[20]: 1091.5

Shows the frequency of the two classes Calculates appropriate weights to reach an equal weighting With the calculated weights, both classes gain equal weight Third is the creation of the DNN model with Keras and the training of the model on the complete data set. The baseline performance in-sample is around 60%: In [21]: import random import tensorflow as tf from keras.layers import Dense from keras.models import Sequential from keras.optimizers import Adam from sklearn.metrics import accuracy_score Using TensorFlow backend. In [22]: def set_seeds(seed=100): random.seed(seed) np.random.seed(seed) tf.random.set_seed(seed) In [23]: optimizer = Adam(lr=0.001) In [24]: def create_model(hl=1, hu=128, optimizer=optimizer): model = Sequential() model.add(Dense(hu, input_dim=len(cols), activation='relu')) for _ in range(hl): model.add(Dense(hu, activation='relu')) model.add(Dense(1, activation='sigmoid')) model.compile(loss='binary_crossentropy',

Baseline Prediction

|

215

optimizer=optimizer, metrics=['accuracy']) return model In [25]: set_seeds() model = create_model(hl=1, hu=128) In [26]: %%time model.fit(data[cols], data['d'], epochs=50, verbose=False, class_weight=cw(data)) CPU times: user 6.44 s, sys: 939 ms, total: 7.38 s Wall time: 4.07 s Out[26]: In [27]: model.evaluate(data[cols], data['d']) 2183/2183 [==============================] - 0s 24us/step Out[27]: [0.582192026280068, 0.6087952256202698] In [28]: data['p'] = np.where(model.predict(data[cols]) > 0.5, 1, 0) In [29]: data['p'].value_counts() Out[29]: 1 1340 0 843 Name: p, dtype: int64

Python random number seed NumPy random number seed TensorFlow random number seed

Default optimizer (see https://oreil.ly/atpu8) First layer Additional layers Output layer Loss function (see https://oreil.ly/cVGVf) Optimizer to be used Additional metrics to be collected

216

|

Chapter 7: Dense Neural Networks

The same holds true for the performance of the model out-of-sample. It is still well above 60%. This can be considered already quite good: In [30]: split = int(len(data) * 0.8) In [31]: train = data.iloc[:split].copy() In [32]: test = data.iloc[split:].copy() In [33]: set_seeds() model = create_model(hl=1, hu=128) In [34]: %%time model.fit(train[cols], train['d'], epochs=50, verbose=False, validation_split=0.2, shuffle=False, class_weight=cw(train)) CPU times: user 4.72 s, sys: 686 ms, total: 5.41 s Wall time: 3.14 s Out[34]: In [35]: model.evaluate(train[cols], train['d']) 1746/1746 [==============================] - 0s 13us/step Out[35]: [0.612861613500842, 0.5853379368782043] In [36]: model.evaluate(test[cols], test['d']) 437/437 [==============================] - 0s 16us/step Out[36]: [0.5946959675858714, 0.6247139573097229] In [37]: test['p'] = np.where(model.predict(test[cols]) > 0.5, 1, 0) In [38]: test['p'].value_counts() Out[38]: 1 291 0 146 Name: p, dtype: int64

Splits the whole data set… …into the training data set… …and the test data set. Evaluates the in-sample performance. Evaluates the out-of-sample performance.

Baseline Prediction

|

217

Figure 7-2 shows how the accuracy on the training and validation data sub-sets changes over the training epochs: In [39]: res = pd.DataFrame(model.history.history) In [40]: res[['accuracy', 'val_accuracy']].plot(figsize=(10, 6), style='--');

Figure 7-2. Training and validation accuracy values The analysis in this section sets the stage for the more elaborate use of DNNs with Keras. It presents a baseline market prediction approach. The following sections add different elements that are primarily supposed to improve the out-of-sample model performance and to avoid overfitting of the model to the training data.

Normalization The baseline prediction in “Baseline Prediction” on page 214 takes the lagged features as they are. In Chapter 6, the features data is normalized by subtracting the mean of the training data for every feature and dividing it by the standard deviation of the training data. This normalization technique is called Gaussian normalization and proves often, if not always, to be an important aspect when training a neural network. As the following Python code and its results illustrate, the in-sample performance increases significantly when working with normalized features data. The out-ofsample performance also slightly increases. However, there is no guarantee that the out-of-sample performance increases through features normalization: In [41]: mu, std = train.mean(), train.std() In [42]: train_ = (train - mu) / std In [43]: set_seeds()

218

|

Chapter 7: Dense Neural Networks

model = create_model(hl=2, hu=128) In [44]: %%time model.fit(train_[cols], train['d'], epochs=50, verbose=False, validation_split=0.2, shuffle=False, class_weight=cw(train)) CPU times: user 5.81 s, sys: 879 ms, total: 6.69 s Wall time: 3.53 s Out[44]: In [45]: model.evaluate(train_[cols], train['d']) 1746/1746 [==============================] - 0s 14us/step Out[45]: [0.4253406366728084, 0.887170672416687] In [46]: test_ = (test - mu) / std In [47]: model.evaluate(test_[cols], test['d']) 437/437 [==============================] - 0s 24us/step Out[47]: [1.1377735263422917, 0.681922197341919] In [48]: test['p'] = np.where(model.predict(test_[cols]) > 0.5, 1, 0) In [49]: test['p'].value_counts() Out[49]: 0 281 1 156 Name: p, dtype: int64

Calculates the mean and standard deviation for all training features Normalizes the training data set based on Gaussian normalization Evaluates the in-sample performance Normalizes the test data set based on Gaussian normalization Evaluates the out-of-sample performance A major problem that often arises is overfitting. It is impressively visualized in Figure 7-3, which shows a steadily improving training accuracy while the validation accuracy decreases slowly: In [50]: res = pd.DataFrame(model.history.history) In [51]: res[['accuracy', 'val_accuracy']].plot(figsize=(10, 6), style='--');

Normalization

|

219

Figure 7-3. Training and validation accuracy values (normalized features data) Three candidate methods to avoid overfitting are dropout, regularization, and bagging. The following sections discuss these methods. The impact of the chosen optimizer is also discussed later in this chapter.

Dropout The idea of dropout is that neural networks should not use all hidden units during the training stage. The analogy to the human brain is that a human being regularly for‐ gets information that was previously learned. This, so to say, keeps the human brain “open minded.” Ideally, a neural network should behave similarly: the connections in the DNN should not become too strong in order to avoid overfitting to the training data. Technically, a Keras model has additional layers between the hidden layers that man‐ age the dropout. The major parameter is the rate with which the hidden units of a layer get dropped. These drops in general happen in randomized fashion. This can be avoided by fixing the seed parameter. While the in-sample performance decreases, the out-of-sample performance slightly decreases as well. However, the difference between the two performance measures is smaller, which is in general a desirable situation: In [52]: from keras.layers import Dropout In [53]: def create_model(hl=1, hu=128, dropout=True, rate=0.3, optimizer=optimizer): model = Sequential() model.add(Dense(hu, input_dim=len(cols), activation='relu'))

220

|

Chapter 7: Dense Neural Networks

if dropout: model.add(Dropout(rate, seed=100)) for _ in range(hl): model.add(Dense(hu, activation='relu')) if dropout: model.add(Dropout(rate, seed=100)) model.add(Dense(1, activation='sigmoid')) model.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy']) return model In [54]: set_seeds() model = create_model(hl=1, hu=128, rate=0.3) In [55]: %%time model.fit(train_[cols], train['d'], epochs=50, verbose=False, validation_split=0.15, shuffle=False, class_weight=cw(train)) CPU times: user 5.46 s, sys: 758 ms, total: 6.21 s Wall time: 3.53 s Out[55]: In [56]: model.evaluate(train_[cols], train['d']) 1746/1746 [==============================] - 0s 20us/step Out[56]: [0.4423361133190911, 0.7840778827667236] In [57]: model.evaluate(test_[cols], test['d']) 437/437 [==============================] - 0s 34us/step Out[57]: [0.5875822428434883, 0.6430205702781677]

Adds dropout after each layer As Figure 7-4 illustrates, the training accuracy and validation accuracy now do not drift apart as fast as before: In [58]: res = pd.DataFrame(model.history.history) In [59]: res[['accuracy', 'val_accuracy']].plot(figsize=(10, 6), style='--');

Dropout

|

221

Figure 7-4. Training and validation accuracy values (with dropout)

Intentional Forgetting Dropout in the Sequential model of Keras emulates what all human beings experience: forgetting previously memorized infor‐ mation. This is accomplished by deactivating certain hidden units of a hidden layer during training. In effect, this often avoids, to a larger extent, overfitting a neural network to the training data.

Regularization Another means to avoid overfitting is regularization. With regularization, large weights in the neural network get penalized in the calculation of the loss (function). This avoids the situation where certain connections in the DNN become too strong and dominant. Regularization can be introduced in a Keras DNN through a parame‐ ter in the Dense layers. Depending on the regularization parameter chosen, training and test accuracy can be kept quite close together. Two regularizers are in general used, one based on the linear norm, l1, and one based on the Euclidean norm, l2. The following Python code adds regularization to the model creation function: In [60]: from keras.regularizers import l1, l2 In [61]: def create_model(hl=1, hu=128, dropout=False, rate=0.3, regularize=False, reg=l1(0.0005), optimizer=optimizer, input_dim=len(cols)): if not regularize: reg = None model = Sequential() model.add(Dense(hu, input_dim=input_dim,

222

|

Chapter 7: Dense Neural Networks

activity_regularizer=reg, activation='relu')) if dropout: model.add(Dropout(rate, seed=100)) for _ in range(hl): model.add(Dense(hu, activation='relu', activity_regularizer=reg)) if dropout: model.add(Dropout(rate, seed=100)) model.add(Dense(1, activation='sigmoid')) model.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy']) return model In [62]: set_seeds() model = create_model(hl=1, hu=128, regularize=True) In [63]: %%time model.fit(train_[cols], train['d'], epochs=50, verbose=False, validation_split=0.2, shuffle=False, class_weight=cw(train)) CPU times: user 5.49 s, sys: 1.05 s, total: 6.54 s Wall time: 3.15 s Out[63]: In [64]: model.evaluate(train_[cols], train['d']) 1746/1746 [==============================] - 0s 15us/step Out[64]: [0.5307255412568205, 0.7691867351531982] In [65]: model.evaluate(test_[cols], test['d']) 437/437 [==============================] - 0s 22us/step Out[65]: [0.8428352184644826, 0.6590389013290405]

Regularization is added to each layer. Figure 7-5 shows the training and validation accuracy under regularization. The two performance measures are much closer together than previously seen: In [66]: res = pd.DataFrame(model.history.history) In [67]: res[['accuracy', 'val_accuracy']].plot(figsize=(10, 6), style='--');

Regularization

|

223

Figure 7-5. Training and validation accuracy values (with regularization) Of course, dropout and regularization can be used together. The idea is that the two measures combined even better avoid overfitting and bring the in-sample and out-ofsample accuracy values closer together. And indeed the difference between the two measures is lowest in this case: In [68]: set_seeds() model = create_model(hl=2, hu=128, dropout=True, rate=0.3, regularize=True, reg=l2(0.001), ) In [69]: %%time model.fit(train_[cols], train['d'], epochs=50, verbose=False, validation_split=0.2, shuffle=False, class_weight=cw(train)) CPU times: user 7.06 s, sys: 958 ms, total: 8.01 s Wall time: 4.28 s Out[69]: In [70]: model.evaluate(train_[cols], train['d']) 1746/1746 [==============================] - 0s 18us/step Out[70]: [0.5007762827004764, 0.7691867351531982] In [71]: model.evaluate(test_[cols], test['d']) 437/437 [==============================] - 0s 23us/step Out[71]: [0.6191965124699835, 0.6864988803863525]

224

|

Chapter 7: Dense Neural Networks

Dropout is added to the model creation. Regularization is added to the model creation. Figure 7-6 shows the training and validation accuracy when combining dropout with regularization. The difference between training and validation data accuracy over the training epochs is some four percentage points only on average: In [72]: res = pd.DataFrame(model.history.history) In [73]: res[['accuracy', 'val_accuracy']].plot(figsize=(10, 6), style='--');

Figure 7-6. Training and validation accuracy values (with dropout and regularization)

Penalizing Large Weights Regularization avoids overfitting by penalizing large weights in a neural network. Single weights cannot get that large enough to dominate a neural network. The penalties keep weights on a com‐ parable level.

Bagging The bagging method to avoid overfitting is already used in Chapter 6, although only for the scikit-learn MLPRegressor model. There is also a wrapper for a Keras DNN classification model to expose it in scikit-learn fashion, namely the KerasClassi fier class. The following Python code combines the Keras DNN modeling based on the wrapper with the BaggingClassifier from scikit-learn. The in-sample and out-of-sample performance measures are relatively high, around 70%. However, the

Bagging

|

225

result is driven by the class imbalance, as addressed previously, and as reflected here in the high frequency of the 0 predictions: In [75]: from sklearn.ensemble import BaggingClassifier from keras.wrappers.scikit_learn import KerasClassifier In [76]: max_features = 0.75 In [77]: set_seeds() base_estimator = KerasClassifier(build_fn=create_model, verbose=False, epochs=20, hl=1, hu=128, dropout=True, regularize=False, input_dim=int(len(cols) * max_features)) In [78]: model_bag = BaggingClassifier(base_estimator=base_estimator, n_estimators=15, max_samples=0.75, max_features=max_features, bootstrap=True, bootstrap_features=True, n_jobs=1, random_state=100, ) In [79]: %time model_bag.fit(train_[cols], train['d']) CPU times: user 40 s, sys: 5.23 s, total: 45.3 s Wall time: 26.3 s Out[79]: BaggingClassifier(base_estimator=, bootstrap_features=True, max_features=0.75, max_samples=0.75, n_estimators=15, n_jobs=1, random_state=100) In [80]: model_bag.score(train_[cols], train['d']) Out[80]: 0.720504009163803 In [81]: model_bag.score(test_[cols], test['d']) Out[81]: 0.6704805491990846 In [82]: test['p'] = model_bag.predict(test_[cols]) In [83]: test['p'].value_counts() Out[83]: 0 408 1 29 Name: p, dtype: int64

The base estimator, here a Keras Sequential model, is instantiated. The BaggingClassifier model is instantiated for a number of equal base estimators.

226

| Chapter 7: Dense Neural Networks

Distributing Learning Bagging, in a sense, distributes learning among a number of neural networks (or other models) in that each neural network, for exam‐ ple, only sees certain parts of the training data set and only a selec‐ tion of the features. This avoids the risk that a single neural network overfits the complete training data set. The prediction is based on all selectively trained neural networks together.

Optimizers The Keras package offers a selection of optimizers that can be used in combination with the Sequential model (see https://oreil.ly/atpu8). Different optimizers might show different performances, with regard to both the time the training takes and the prediction accuracy. The following Python code uses different optimizers and bench‐ marks their performance. In all cases, the default parametrization of Keras is used. The out-of-sample performance does not vary that much. However, the in-sample performance, given the different optimizers, varies by a wide margin: In [84]: import time In [85]: optimizers = ['sgd', 'rmsprop', 'adagrad', 'adadelta', 'adam', 'adamax', 'nadam'] In [86]: %%time for optimizer in optimizers: set_seeds() model = create_model(hl=1, hu=128, dropout=True, rate=0.3, regularize=False, reg=l2(0.001), optimizer=optimizer ) t0 = time.time() model.fit(train_[cols], train['d'], epochs=50, verbose=False, validation_split=0.2, shuffle=False, class_weight=cw(train)) t1 = time.time() t = t1 - t0 acc_tr = model.evaluate(train_[cols], train['d'], verbose=False)[1] acc_te = model.evaluate(test_[cols], test['d'], verbose=False)[1] out = f'{optimizer:10s} | time[s]: {t:.4f} | in-sample={acc_tr:.4f}' out += f' | out-of-sample={acc_te:.4f}' print(out) sgd | time[s]: 2.8092 | in-sample=0.6363 | out-of-sample=0.6568 rmsprop | time[s]: 2.9480 | in-sample=0.7600 | out-of-sample=0.6613 adagrad | time[s]: 2.8472 | in-sample=0.6747 | out-of-sample=0.6499 adadelta | time[s]: 3.2068 | in-sample=0.7279 | out-of-sample=0.6522 adam | time[s]: 3.2364 | in-sample=0.7365 | out-of-sample=0.6545

Optimizers

|

227

adamax nadam CPU times: Wall time:

| time[s]: 3.2465 | in-sample=0.6982 | out-of-sample=0.6476 | time[s]: 4.1275 | in-sample=0.7944 | out-of-sample=0.6590 user 35.9 s, sys: 4.55 s, total: 40.4 s 23.1 s

Instantiates the DNN model for the given optimizer Fits the model with the given optimizer Evaluates the in-sample performance Evaluates the out-of-sample performance

Conclusions This chapter dives deeper into the world of DNNs and uses Keras as the primary package. Keras offers a high degree of flexibility in composing DNNs. The results in this chapter are promising in that both in-sample and out-of-sample performance— with regard to the prediction accuracy—are consistently 60% and higher. However, prediction accuracy is just one side of the coin. An appropriate trading strategy must be available and implementable to economically profit from the predictions, or “signals.” This topic of paramount importance in the context of algorithmic trading is discussed in detail in Part IV. The next two chapters first illustrate the use of different neural networks (recurrent and convolutional neural networks) and learning techni‐ ques (reinforcement learning).

References Keras is a powerful and comprehensive package for deep learning with TensforFlow as its primary backend. The project is also evolving fast. Make sure to stay up to date via the main project page. The major resources about Keras in book form are the fol‐ lowing:

Chollet, Francois. 2017. Deep Learning with Python. Shelter Island: Manning. Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. Cam‐ bridge: MIT Press. http://deeplearningbook.org.

228

| Chapter 7: Dense Neural Networks

CHAPTER 8

Recurrent Neural Networks

History never repeats itself, but it rhymes. —Mark Twain (probably) My life seemed to be a series of events and accidents. Yet when I look back, I see a pat‐ tern. —Bernoît Mandelbrot

This chapter is about recurrent neural networks (RNNs). This type of network is specifically designed to learn about sequential data, such as text or time series data. The discussion in this chapter takes, as before, a practical approach and relies mainly on worked-out Python examples, making use of Keras.1 “First Example” on page 230 and “Second Example” on page 234 introduce RNNs on the basis of two simple examples with sample numerical data. The application of RNNs to predict sequential data is illustrated. “Financial Price Series” on page 237 then works with financial price series data and applies the RNN approach to predict such a series directly via estimation. “Financial Return Series” on page 240 then works with returns data to predict the future direction of the price of a financial instrument also via an estimation approach. “Financial Features” on page 242 adds financial features to the mix—in addition to price and return data—to predict the market direction. Three different approaches are illustrated in this section: prediction via a shallow RNN for both estimation and classification, as well as prediction via a deep RNN for classification.

1 For technical details of RNNs, refer to Goodfellow et al. (2016, ch. 10). For the practical implementation, refer

to Chollet (2017, ch. 6).

229

The chapter shows that the application of RNNs to financial time series data can ach‐ ieve a prediction accuracy of well above 60% out-of-sample in the context of direc‐ tional market predictions. However, the results obtained cannot fully keep up with those seen in Chapter 7. This might come as a surprise, since RNNs are meant to work well with financial time series data, which is the primary focus of this book.

First Example To illustrate the training and usage of RNNs, consider a simple example based on a sequence of integers. First, some imports and configurations: In [1]: import os import random import numpy as np import pandas as pd import tensorflow as tf from pprint import pprint from pylab import plt, mpl plt.style.use('seaborn') mpl.rcParams['savefig.dpi'] = 300 mpl.rcParams['font.family'] = 'serif' pd.set_option('precision', 4) np.set_printoptions(suppress=True, precision=4) os.environ['PYTHONHASHSEED'] = '0' In [2]: def set_seeds(seed=100): random.seed(seed) np.random.seed(seed) tf.random.set_seed(seed) set_seeds()

Function to set all seed values Second is the simple data set that is transformed into an appropriate shape: In [3]: a = np.arange(100) a Out[3]: array([ 0, 1, 2, 17, 18, 19, 34, 35, 36, 51, 52, 53, 68, 69, 70, 85, 86, 87,

3, 20, 37, 54, 71, 88,

4, 21, 38, 55, 72, 89,

In [4]: a = a.reshape((len(a), -1)) In [5]: a.shape Out[5]: (100, 1) In [6]: a[:5] Out[6]: array([[0],

230

|

Chapter 8: Recurrent Neural Networks

5, 22, 39, 56, 73, 90,

6, 23, 40, 57, 74, 91,

7, 24, 41, 58, 75, 92,

8, 25, 42, 59, 76, 93,

9, 26, 43, 60, 77, 94,

10, 27, 44, 61, 78, 95,

11, 28, 45, 62, 79, 96,

12, 29, 46, 63, 80, 97,

13, 30, 47, 64, 81, 98,

14, 15, 31, 32, 48, 49, 65, 66, 82, 83, 99])

16, 33, 50, 67, 84,

[1], [2], [3], [4]])

Sample data Reshaping to two dimensions Using the TimeseriesGenerator, the raw data can be transformed into an object suited for the training of an RNN. The idea is to use a certain number of lags of the original data to train the model to predict the next value in the sequence. For exam‐ ple, 0, 1, 2 are the three lagged values (features) used to predict the value 3 (label). In the same way, 1, 2, 3 are used to predict 4: In [7]: from keras.preprocessing.sequence import TimeseriesGenerator Using TensorFlow backend. In [8]: lags = 3 In [9]: g = TimeseriesGenerator(a, a, length=lags, batch_size=5) In [10]: pprint(list(g)[0]) (array([[[0], [1], [2]], [[1], [2], [3]], [[2], [3], [4]], [[3], [4], [5]], [[4], [5], [6]]]), array([[3], [4], [5], [6], [7]]))

TimeseriesGenerator creates batches of lagged sequential data.

First Example

|

231

The creation of the RNN model is similar to DNNs. The following Python code uses a single hidden layer of type SimpleRNN (Chollet 2017, ch. 6; also see Keras recurrent layers). Even with relatively few hidden units, the number of trainable parameters is quite large. The .fit_generator() method takes as input generator objects such as those created with TimeseriesGenerator: In [11]: from keras.models import Sequential from keras.layers import SimpleRNN, LSTM, Dense In [12]: model = Sequential() model.add(SimpleRNN(100, activation='relu', input_shape=(lags, 1))) model.add(Dense(1, activation='linear')) model.compile(optimizer='adagrad', loss='mse', metrics=['mae']) In [13]: model.summary() Model: "sequential_1" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= simple_rnn_1 (SimpleRNN) (None, 100) 10200 _________________________________________________________________ dense_1 (Dense) (None, 1) 101 ================================================================= Total params: 10,301 Trainable params: 10,301 Non-trainable params: 0 _________________________________________________________________ In [14]: %%time model.fit_generator(g, epochs=1000, steps_per_epoch=5, verbose=False) CPU times: user 17.4 s, sys: 3.9 s, total: 21.3 s Wall time: 30.8 s Out[14]:

The single hidden layer is of type SimpleRNN. The summary of the shallow RNN. The fitting of the RNN based on the generator object.

232

|

Chapter 8: Recurrent Neural Networks

The performance metrics might show relatively erratic behavior when training RNNs (see Figure 8-1): In [15]: res = pd.DataFrame(model.history.history) In [16]: res.tail(3) Out[16]: loss mae 997 0.0001 0.0109 998 0.0007 0.0211 999 0.0001 0.0101 In [17]: res.iloc[10:].plot(figsize=(10, 6), style=['--', '--']);

Figure 8-1. Performance metrics during RNN training Having a trained RNN available, the following Python code generates in-sample and out-of-sample predictions: In [18]: x = np.array([21, 22, 23]).reshape((1, lags, 1)) y = model.predict(x, verbose=False) int(round(y[0, 0])) Out[18]: 24 In [19]: x = np.array([87, 88, 89]).reshape((1, lags, 1)) y = model.predict(x, verbose=False) int(round(y[0, 0])) Out[19]: 90 In [20]: x = np.array([187, 188, 189]).reshape((1, lags, 1)) y = model.predict(x, verbose=False) int(round(y[0, 0])) Out[20]: 190 In [21]: x = np.array([1187, 1188, 1189]).reshape((1, lags, 1))

First Example

|

233

y = model.predict(x, verbose=False) int(round(y[0, 0])) Out[21]: 1194

In-sample prediction Out-of-sample prediction Far-out-of-sample prediction Even for far-out-of-sample predictions, the results are good in general in this simple case. However, the problem at hand could, for example, be perfectly solved by the application of OLS regression. Therefore, the effort involved for the training of an RNN for such a problem is quite high given the performance of the RNN.

Second Example The first example illustrates the training of an RNN for a simple problem that is easy to solve not only by OLS regression but also by a human being inspecting the data. The second example is a bit more challenging. The input data is transformed by a quadratic term and a trigonometric term, as well as by adding white noise to it. Figure 8-2 shows the resulting sequence for the interval − 2π, 2π]: In [22]: def transform(x): y = 0.05 * x ** 2 + 0.2 * x + np.sin(x) + 5 y += np.random.standard_normal(len(x)) * 0.2 return y In [23]: x = np.linspace(-2 * np.pi, 2 * np.pi, 500) a = transform(x) In [24]: plt.figure(figsize=(10, 6)) plt.plot(x, a);

Deterministic transformation Stochastic transformation

234

|

Chapter 8: Recurrent Neural Networks

Figure 8-2. Sample sequence data As before, the raw data is reshaped, TimeseriesGenerator is applied, and the RNN with a single hidden layer is trained: In [25]: a = a.reshape((len(a), -1)) In [26]: a[:5] Out[26]: array([[5.6736], [5.68 ], [5.3127], [5.645 ], [5.7118]]) In [27]: lags = 5 In [28]: g = TimeseriesGenerator(a, a, length=lags, batch_size=5) In [29]: model = Sequential() model.add(SimpleRNN(500, activation='relu', input_shape=(lags, 1))) model.add(Dense(1, activation='linear')) model.compile(optimizer='rmsprop', loss='mse', metrics=['mae']) In [30]: model.summary() Model: "sequential_2" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= simple_rnn_2 (SimpleRNN) (None, 500) 251000 _________________________________________________________________ dense_2 (Dense) (None, 1) 501 ================================================================= Total params: 251,501 Trainable params: 251,501

Second Example

|

235

Non-trainable params: 0 _________________________________________________________________ In [31]: %%time model.fit_generator(g, epochs=500, steps_per_epoch=10, verbose=False) CPU times: user 1min 6s, sys: 14.6 s, total: 1min 20s Wall time: 23.1 s Out[31]:

The following Python code predicts sequence values for the interval − 6π, 6π]. This interval is three times the size of the training interval and contains out-of-sample pre‐ dictions both on the left-hand side and on the right-hand side of the training interval. Figure 8-3 shows that the model performs quite well, even out-of-sample: In [32]: x = np.linspace(-6 * np.pi, 6 * np.pi, 1000) d = transform(x) In [33]: g_ = TimeseriesGenerator(d, d, length=lags, batch_size=len(d)) In [34]: f = list(g_)[0][0].reshape((len(d) - lags, lags, 1)) In [35]: y = model.predict(f, verbose=False) In [36]: plt.figure(figsize=(10, 6)) plt.plot(x[lags:], d[lags:], label='data', alpha=0.75) plt.plot(x[lags:], y, 'r.', label='pred', ms=3) plt.axvline(-2 * np.pi, c='g', ls='--') plt.axvline(2 * np.pi, c='g', ls='--') plt.text(-15, 22, 'out-of-sample') plt.text(-2, 22, 'in-sample') plt.text(10, 22, 'out-of-sample') plt.legend();

Enlarges the sample data set In-sample and out-of-sample prediction

Simplicity of Examples The first two examples are deliberately chosen to be simple. Both problems posed in the examples can be solved more efficiently with OLS regression, for example, by allowing for trigonometric basis functions in the second example. However, the training of RNNs for nontrivial sequence data, such as financial time series data, is basically the same. In such a context, OLS regression, for instance, can in general not keep up with the capabilities of RNNs.

236

|

Chapter 8: Recurrent Neural Networks

Figure 8-3. In-sample and out-of-sample predictions of the RNN

Financial Price Series As a first application of RNNs to financial time series data, consider intraday EUR/USD quotes. With the approach introduced in the previous two sections, the training of the RNN on the financial time series is straightforward. First, the data is imported and resampled. The data is also normalized and transformed into the appropriate ndarray object: In [37]: url = 'http://hilpisch.com/aiif_eikon_id_eur_usd.csv' In [38]: symbol = 'EUR_USD' In [39]: raw = pd.read_csv(url, index_col=0, parse_dates=True) In [40]: def generate_data(): data = pd.DataFrame(raw['CLOSE']) data.columns = [symbol] data = data.resample('30min', label='right').last().ffill() return data In [41]: data = generate_data() In [42]: data = (data - data.mean()) / data.std() In [43]: p = data[symbol].values In [44]: p = p.reshape((len(p), -1))

Financial Price Series

|

237

Selects a single column Renames the column Resamples the data Applies Gaussian normalization Reshapes the data set to two dimensions Second, the RNN is trained based on the generator object. The function create_rnn_model() allows the creation of an RNN with a SimpleRNN or an LSTM (long short-term memory) layer (Chollet 2017, ch. 6; also see Keras recurrent layers). In [45]: lags = 5 In [46]: g = TimeseriesGenerator(p, p, length=lags, batch_size=5) In [47]: def create_rnn_model(hu=100, lags=lags, layer='SimpleRNN', features=1, algorithm='estimation'): model = Sequential() if layer is 'SimpleRNN': model.add(SimpleRNN(hu, activation='relu', input_shape=(lags, features))) else: model.add(LSTM(hu, activation='relu', input_shape=(lags, features))) if algorithm == 'estimation': model.add(Dense(1, activation='linear')) model.compile(optimizer='adam', loss='mse', metrics=['mae']) else: model.add(Dense(1, activation='sigmoid')) model.compile(optimizer='adam', loss='mse', metrics=['accuracy']) return model In [48]: model = create_rnn_model() In [49]: %%time model.fit_generator(g, epochs=500, steps_per_epoch=10, verbose=False) CPU times: user 20.8 s, sys: 4.66 s, total: 25.5 s Wall time: 11.2 s Out[49]:

Adds a SimpleRNN layer or LSTM layer Adds an output layer for estimation or classification

238

|

Chapter 8: Recurrent Neural Networks

Third, the in-sample prediction is generated. As Figure 8-4 illustrates, the RNN is capable of capturing the structure of the normalized financial time series data. Based on this visualization, the prediction accuracy seems quite good: In [50]: y = model.predict(g, verbose=False) In [51]: data['pred'] = np.nan data['pred'].iloc[lags:] = y.flatten() In [52]: data[[symbol, 'pred']].plot( figsize=(10, 6), style=['b', 'r-.'], alpha=0.75);

Figure 8-4. In-sample prediction for financial price series by the RNN (whole data set) However, the visualization suggests a result that does not hold up upon closer inspec‐ tion. Figure 8-5 zooms in and only shows 50 data points from the original data set and of the prediction. It becomes clear that the prediction values from the RNN are basically just the most previous lag, shifted by one time interval. Visually speaking, the prediction line is the financial time series itself, moved one time interval to the right: In [53]: data[[symbol, 'pred']].iloc[50:100].plot( figsize=(10, 6), style=['b', 'r-.'], alpha=0.75);

Financial Price Series

|

239

Figure 8-5. In-sample prediction for financial price series by the RNN (data sub-set)

RNNs and Efficient Markets The results for the prediction of a financial price series based on an RNN are in line with the OLS regression approach used in Chap‐ ter 6 to illustrate the EMH. There, it is illustrated that, in a leastsquares sense, today’s price is the best predictor for tomorrow’s price. The application of an RNN to price data does not yield any other insight.

Financial Return Series As previous analyses have shown, it might be easier to predict returns instead of pri‐ ces. Therefore, the following Python code repeats the preceding analysis based on log returns: In [54]: data = generate_data() In [55]: data['r'] = np.log(data / data.shift(1)) In [56]: data.dropna(inplace=True) In [57]: data = (data - data.mean()) / data.std() In [58]: r = data['r'].values In [59]: r = r.reshape((len(r), -1)) In [60]: g = TimeseriesGenerator(r, r, length=lags, batch_size=5)

240

|

Chapter 8: Recurrent Neural Networks

In [61]: model = create_rnn_model() In [62]: %%time model.fit_generator(g, epochs=500, steps_per_epoch=10, verbose=False) CPU times: user 20.4 s, sys: 4.2 s, total: 24.6 s Wall time: 11.3 s Out[62]:

As Figure 8-6 shows, the RNN’s predictions are not too good in absolute terms. How‐ ever, they seem to get the market direction (sign of the return) somehow right: In [63]: y = model.predict(g, verbose=False) In [64]: data['pred'] = np.nan data['pred'].iloc[lags:] = y.flatten() data.dropna(inplace=True) In [65]: data[['r', 'pred']].iloc[50:100].plot( figsize=(10, 6), style=['b', 'r-.'], alpha=0.75); plt.axhline(0, c='grey', ls='--')

Figure 8-6. In-sample prediction for financial return series by the RNN (data sub-set) While Figure 8-6 only provides an indication, the relatively high accuracy score sup‐ ports the assumption that the RNN might perform better on a return than on a price series: In [66]: from sklearn.metrics import accuracy_score

Financial Return Series

|

241

In [67]: accuracy_score(np.sign(data['r']), np.sign(data['pred'])) Out[67]: 0.6806532093445226

However, to get a realistic picture, a train-test split is in order. The accuracy score out-of-sample is not as high as the one seen for the whole data set in-sample, but it is still high for the problem at hand: In [68]: split = int(len(r) * 0.8) In [69]: train = r[:split] In [70]: test = r[split:] In [71]: g = TimeseriesGenerator(train, train, length=lags, batch_size=5) In [72]: set_seeds() model = create_rnn_model(hu=100) In [73]: %%time model.fit_generator(g, epochs=100, steps_per_epoch=10, verbose=False) CPU times: user 5.67 s, sys: 1.09 s, total: 6.75 s Wall time: 2.95 s Out[73]: In [74]: g_ = TimeseriesGenerator(test, test, length=lags, batch_size=5) In [75]: y = model.predict(g_) In [76]: accuracy_score(np.sign(test[lags:]), np.sign(y)) Out[76]: 0.6708428246013668

Splits the data into train and test data sub-sets Fits the model on the training data Tests the model on the testing data

Financial Features The application of RNNs is not restricted to the raw price or return data. Additional features can also be included to improve the prediction of the RNN. The following Python code adds typical financial features to the data set: In [77]: data = generate_data() In [78]: data['r'] = np.log(data / data.shift(1)) In [79]: window = 20 data['mom'] = data['r'].rolling(window).mean()

242

|

Chapter 8: Recurrent Neural Networks

data['vol'] = data['r'].rolling(window).std() In [80]: data.dropna(inplace=True)

Adds a time series momentum feature Adds a rolling volatility feature

Estimation The out-of-sample accuracy, maybe somewhat surprisingly, drops significantly in the estimation case. In other words, there is no improvement observed from adding financial features in this particular case: In [81]: split = int(len(data) * 0.8) In [82]: train = data.iloc[:split].copy() In [83]: mu, std = train.mean(), train.std() In [84]: train = (train - mu) / std In [85]: test = data.iloc[split:].copy() In [86]: test = (test - mu) / std In [87]: g = TimeseriesGenerator(train.values, train['r'].values, length=lags, batch_size=5) In [88]: set_seeds() model = create_rnn_model(hu=100, features=len(data.columns), layer='SimpleRNN') In [89]: %%time model.fit_generator(g, epochs=100, steps_per_epoch=10, verbose=False) CPU times: user 5.24 s, sys: 1.08 s, total: 6.32 s Wall time: 2.73 s Out[89]: In [90]: g_ = TimeseriesGenerator(test.values, test['r'].values, length=lags, batch_size=5) In [91]: y = model.predict(g_).flatten() In [92]: accuracy_score(np.sign(test['r'].iloc[lags:]), np.sign(y)) Out[92]: 0.37299771167048057

Financial Features

|

243

Calculates the first and second moment of the training data Applies Gaussian normalization to the training data Applies Gaussian normalization to the testing data—based on the statistics from the training data Fits the model on the training data Tests the model on the testing data

Classification The analyses so far use a Keras RNN model for estimation to predict the future direc‐ tion of the price of the financial instrument. The problem at hand is probably better cast directly into a classification setting. The following Python code works with binary labels data and predicts the direction of the price movement directly. It also works this time with an LSTM layer. The out-of-sample accuracy is quite high even for a relatively small number of hidden units and only a few training epochs. The approach again takes class imbalance into account by adjusting the class weights appropriately. The prediction accuracy is quite high in this case with around 65%: In [93]: set_seeds() model = create_rnn_model(hu=50, features=len(data.columns), layer='LSTM', algorithm='classification') In [94]: train_y = np.where(train['r'] > 0, 1, 0) In [95]: np.bincount(train_y) Out[95]: array([2374, 1142]) In [96]: def cw(a): c0, c1 = np.bincount(a) w0 = (1 / c0) * (len(a)) / 2 w1 = (1 / c1) * (len(a)) / 2 return {0: w0, 1: w1} In [97]: g = TimeseriesGenerator(train.values, train_y, length=lags, batch_size=5) In [98]: %%time model.fit_generator(g, epochs=5, steps_per_epoch=10, verbose=False, class_weight=cw(train_y)) CPU times: user 1.25 s, sys: 159 ms, total: 1.41 s Wall time: 947 ms

244

|

Chapter 8: Recurrent Neural Networks

Out[98]: In [99]: test_y = np.where(test['r'] > 0, 1, 0) In [100]: g_ = TimeseriesGenerator(test.values, test_y, length=lags, batch_size=5) In [101]: y = np.where(model.predict(g_, batch_size=None) > 0.5, 1, 0).flatten() In [102]: np.bincount(y) Out[102]: array([492, 382]) In [103]: accuracy_score(test_y[lags:], y) Out[103]: 0.6498855835240275

RNN model for classification Binary training labels Class frequency for training labels Binary testing labels

Deep RNNs Finally, consider deep RNNs, which are RNNs with multiple hidden layers. They are as easily created as deep DNNs. The only requirement is that for the nonfinal hidden layers, the parameter return_sequences is set to True. The following Python func‐ tion to create a deep RNN also allows for the addition of Dropout layers to potentially avoid overfitting. The prediction accuracy is comparable to the one seen in the previ‐ ous sub-section: In [104]: from keras.layers import Dropout In [105]: def create_deep_rnn_model(hl=2, hu=100, layer='SimpleRNN', optimizer='rmsprop', features=1, dropout=False, rate=0.3, seed=100): if hl 0.5, 1, 0).flatten() In [109]: np.bincount(y) Out[109]: array([550, 324]) In [110]: accuracy_score(test_y[lags:], y) Out[110]: 0.6430205949656751

A minimum of two hidden layers is ensured. The first hidden layer. The Dropout layers. The final hidden layer. The model is built for classification.

Conclusions This chapter introduces RNNs with Keras and illustrates the application of such neu‐ ral networks to financial time series data. On the Python level, working with RNNs is not too different from working with DNNs. One major difference is that the training and test data must necessarily be presented in a sequential form to the respective methods. However, this is made easy by the application of the TimeseriesGenerator

246

| Chapter 8: Recurrent Neural Networks

function, which transforms sequential data into a generator object that Keras RNNs can work with. The examples in this chapter work with both financial price series and financial return series. In addition, financial features, such as time series momentum, can also be added easily. The functions presented for model creation allow, among other things, for one to use SimpleRNN or LSTM layers as well as different optimizers. They also allow one to model estimation and classification problems in the context of shal‐ low and deep neural networks. The out-of-sample prediction accuracy, when predicting market direction, is rela‐ tively high for the classification examples—but it’s not that high and can even be quite low for the estimation examples.

References Books and papers cited in this chapter: Chollet, François. 2017. Deep Learning with Python. Shelter Island: Manning. Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. Cam‐ bridge: MIT Press. http://deeplearningbook.org.

References

|

247

CHAPTER 9

Reinforcement Learning

Like a human, our agents learn for themselves to achieve successful strategies that lead to the greatest long-term rewards. This paradigm of learning by trial-and-error, solely from rewards or punishments, is known as reinforcement learning.1 —DeepMind (2016)

The learning algorithms applied in Chapters 7 and 8 fall into the category of super‐ vised learning. These methods require that there is a data set available with features and labels that allows the algorithms to learn relationships between the features and labels to succeed at estimation or classification tasks. As the simple example in Chap‐ ter 1 illustrates, reinforcement learning (RL) works differently. To begin with, there is no need for a comprehensive data set of features and labels to be given up front. The data is rather generated by the learning agent while interacting with the environment of interest. This chapter covers RL in some detail and introduces fundamental notions, as well as one of the most popular algorithms used in the field: Q-learning (QL). Neural networks are not replaced by RL algorithms; they generally play an important role in this context as well. “Fundamental Notions” on page 250 explains fundamental notions in RL, such as environments, states, and agents. “OpenAI Gym” on page 251 introduces the OpenAI Gym suite of RL environments of which the CartPole environment is used as an example. In this environment, which Chapter 2 introduces and discusses briefly, agents must learn how to balance a pole on a cart by moving the cart to the left or to the right. “Monte Carlo Agent” on page 255 shows how to solve the CartPole prob‐ lem by the use of dimensionality reduction and Monte Carlo simulation. Standard supervised learning algorithms such as DNNs are in general not suited to solve

1 See Deep Reinforcement Learning.

249

problems such as the CartPole one since they lack a notion of delayed reward. This problem is illustrated in “Neural Network Agent” on page 257. “DQL Agent” on page 260 discusses a QL agent that explicitly takes into account delayed rewards and is able to solve the CartPole problem. The same agent is applied in “Simple Finance Gym” on page 264 to a simple financial market environment. Although the agent does not perform too well in this setting, the example shows that QL agents can also learn to trade and to become what is often called a trading bot. To improve the learning of QL agents, “Better Finance Gym” on page 268 presents an improved financial market environment that, among other benefits, allows the use of more than one type of fea‐ ture to describe the state of the environment. Based on this improved environment, “FQL Agent” on page 271 introduces and applies an improved financial QL agent that performs better as a trading bot.

Fundamental Notions This section gives a brief overview of the fundamental notions in RL. Among them are the following: Environment The environment defines the problem at hand. This can be a computer game to be played or a financial market to be traded in. State A state subsumes all relevant parameters that describe the current state of the environment. In a computer game, this might be the whole screen with all its pixels. In a financial market, this might include current and historical price levels or financial indicators such as moving averages, macroeconomic variables, and so on. Agent The term agent subsumes all elements of the RL algorithm that interacts with the environment and that learns from these interactions. In a gaming context, the agent might represent a player playing the game. In a financial context, the agent could represent a trader placing bets on rising or falling markets. Action An agent can choose one action from a (limited) set of allowed actions. In a com‐ puter game, movements to the left or right might be allowed actions, whereas in a financial market, going long or short could be admissible actions. Step

250

Given an action of an agent, the state of the environment is updated. One such update is generally called a step. The concept of a step is general enough to encompass both heterogeneous and homogeneous time intervals between two

|

Chapter 9: Reinforcement Learning

steps. While in computer games, real-time interaction with the game environ‐ ment is simulated by rather short, homogeneous time intervals (“game clock”), a trading bot interacting with a financial market environment could take actions at longer, heterogeneous time intervals, for instance. Reward Depending on the action an agent chooses, a reward (or penalty) is awarded. For a computer game, points are a typical reward. In a financial context, profit (or loss) is a standard reward (or penalty). Target The target specifies what the agent tries to maximize. In a computer game, this in general is the score reached by the agent. For a financial trading bot, this might be the accumulated trading profit. Policy The policy defines which action an agent takes given a certain state of the envi‐ ronment. Given a certain state of a computer game, represented by all the pixels that make up the current scene, the policy might specify that the agent chooses “move right” as the action. A trading bot that observes three price increases in a row might decide, according to its policy, to short the market. Episode An episode is a set of steps from the initial state of the environment until success is achieved or failure is observed. In a game, this is from the start of the game until a win or loss. In the financial world, for example, this is from the beginning of the year to the end of the year or to bankruptcy. Sutton and Barto (2018) provide a detailed introduction to the RL field. The book discusses the preceding notions in detail and illustrates them on the basis of a multi‐ tude of concrete examples. The following sections again choose a practical, implementation-oriented approach to RL. The examples discussed illustrate all of the preceding notions on the basis of Python code.

OpenAI Gym In most of the success stories as presented in Chapter 2, RL plays a dominant role. This has spurred widespread interest in RL as an algorithm. OpenAI is an organiza‐ tion that strives to facilitate research in AI in general and in RL in particular. OpenAI has developed and open sourced a suite of environments, called OpenAI Gym, that allows the training of RL agents via a standardized API. Among the many environments, there is the CartPole environment (or game) that simulates a classical RL problem. A pole is standing upright on a cart, and the goal is to learn a policy to balance the pole on the cart by moving the cart either to the right OpenAI Gym

|

251

or to the left. The state of the environment is given by four parameters, describing the following physical measurements: cart position, cart velocity, pole angle, and pole velocity (at tip). Figure 9-1 shows a visualization of the environment.

Figure 9-1. CartPole environment of OpenAI Gym Consider the following Python code that instantiates an environment object for

CartPole and inspects the observation space. The observation space is a model for the

state of the environment:

In [1]: import os import math import random import numpy as np import pandas as pd from pylab import plt, mpl plt.style.use('seaborn') mpl.rcParams['savefig.dpi'] = 300 mpl.rcParams['font.family'] = 'serif' np.set_printoptions(precision=4, suppress=True) os.environ['PYTHONHASHSEED'] = '0' In [2]: import gym In [3]: env = gym.make('CartPole-v0') In [4]: env.seed(100) env.action_space.seed(100) Out[4]: [100] In [5]: env.observation_space

252

|

Chapter 9: Reinforcement Learning

Out[5]: Box(4,) In [6]: env.observation_space.low.astype(np.float16) Out[6]: array([-4.8 , -inf, -0.419, -inf], dtype=float16) In [7]: env.observation_space.high.astype(np.float16) Out[7]: array([4.8 , inf, 0.419, inf], dtype=float16) In [8]: state = env.reset() In [9]: state Out[9]: array([-0.0163,

0.0238, -0.0392, -0.0148])

The environment object, with fixed seed values The observation space with minimal and maximal values Reset of the environment Initial state: cart position, cart velocity, pole angle, pole angular velocity In the following environment, the allowed actions are described by the action space. In this case there are two, and they are represented by 0 (push cart to the left) and 1 (push cart to the right): In [10]: env.action_space Out[10]: Discrete(2) In [11]: env.action_space.n Out[11]: 2 In [12]: env.action_space.sample() Out[12]: 1 In [13]: env.action_space.sample() Out[13]: 0 In [14]: a = env.action_space.sample() a Out[14]: 1 In [15]: state, reward, done, info = env.step(a) state, reward, done, info Out[15]: (array([-0.0158, 0.2195, -0.0395, -0.3196]), 1.0, False, {})

The action space Random actions sampled from the action space Step forward based on random action OpenAI Gym

|

253

New state of the environment, reward, success/failure, additional information As long as done=False, the agent is still in the game and can choose another action. Success is achieved when the agent reaches a total of 200 steps in a row or a total reward of 200 (reward of 1.0 per step). A failure is observed when the pole on the cart reaches a certain angle that would lead to the pole falling from the cart. In that case, done=True is returned. A simple agent is one that follows a completely random policy: no matter what state is observed, the agent chooses a random action. This is what the following code imple‐ ments. The number of steps the agent can go only depends in such a case on how lucky it is. No learning in the form of updating the policy is taking place: In [16]: env.reset() for e in range(1, 200): a = env.action_space.sample() state, reward, done, info = env.step(a) print(f'step={e:2d} | state={state} | action={a} | reward={reward}') if done and (e + 1) < 200: print('*** FAILED ***') break step= 1 | state=[-0.0423 0.1982 0.0256 -0.2476] | action=1 | reward=1.0 step= 2 | state=[-0.0383 0.0028 0.0206 0.0531] | action=0 | reward=1.0 step= 3 | state=[-0.0383 0.1976 0.0217 -0.2331] | action=1 | reward=1.0 step= 4 | state=[-0.0343 0.0022 0.017 0.0664] | action=0 | reward=1.0 step= 5 | state=[-0.0343 0.197 0.0184 -0.2209] | action=1 | reward=1.0 step= 6 | state=[-0.0304 0.0016 0.0139 0.0775] | action=0 | reward=1.0 step= 7 | state=[-0.0303 0.1966 0.0155 -0.2107] | action=1 | reward=1.0 step= 8 | state=[-0.0264 0.0012 0.0113 0.0868] | action=0 | reward=1.0 step= 9 | state=[-0.0264 0.1962 0.013 -0.2023] | action=1 | reward=1.0 step=10 | state=[-0.0224 0.3911 0.009 -0.4908] | action=1 | reward=1.0 step=11 | state=[-0.0146 0.5861 -0.0009 -0.7807] | action=1 | reward=1.0 step=12 | state=[-0.0029 0.7812 -0.0165 -1.0736] | action=1 | reward=1.0 step=13 | state=[ 0.0127 0.9766 -0.0379 -1.3714] | action=1 | reward=1.0 step=14 | state=[ 0.0323 1.1722 -0.0654 -1.6758] | action=1 | reward=1.0 step=15 | state=[ 0.0557 0.9779 -0.0989 -1.4041] | action=0 | reward=1.0 step=16 | state=[ 0.0753 0.7841 -0.127 -1.1439] | action=0 | reward=1.0 step=17 | state=[ 0.0909 0.5908 -0.1498 -0.8936] | action=0 | reward=1.0 step=18 | state=[ 0.1028 0.7876 -0.1677 -1.2294] | action=1 | reward=1.0 step=19 | state=[ 0.1185 0.9845 -0.1923 -1.5696] | action=1 | reward=1.0 step=20 | state=[ 0.1382 0.7921 -0.2237 -1.3425] | action=0 | reward=1.0 *** FAILED *** In [17]: done Out[17]: True

254

| Chapter 9: Reinforcement Learning

Random action policy Stepping forward one step Failure if less than 200 steps

Data Through Interaction Whereas in supervised learning the training, validation, and test data sets are assumed to exist before the training begins, in RL the agent generates its data itself by interacting with the environment. In many contexts, such as in games, this is a huge simplification. Consider the game of chess: instead of loading thousands of histor‐ ical human-played chess games into a computer, an RL agent can generate thousands or millions of games itself by playing against another chess engine or another version of itself, for instance.

Monte Carlo Agent The CartPole problem does not necessarily require a full-fledged RL approach nor some neural network to be solved. This section presents a simple solution to the problem based on Monte Carlo simulation. To this end, a specific policy is defined that makes use of dimensionality reduction. In that case, the four parameters defining a state of the environment are collapsed, via a linear combination, into a single realvalued parameter.2 The following Python code implements this idea: In [18]: np.random.seed(100) In [19]: weights = np.random.random(4) * 2 - 1 In [20]: weights Out[20]: array([ 0.0868, -0.4433, -0.151 ,

0.6896])

In [21]: state = env.reset() In [22]: state Out[22]: array([-0.0347, -0.0103,

0.047 , -0.0315])

In [23]: s = np.dot(state, weights) s Out[23]: -0.02725361929630797

Random weights for fixed seed value

2 See, for example, this blog post.

Monte Carlo Agent

|

255

Initial state of the environment Dot product of state and weights The policy is then defined based on the sign of the single state parameter s: In [24]: if s < 0: a = 0 else: a = 1 In [25]: a Out[25]: 0

This policy can then be used to play an episode of the CartPole game. Given the ran‐ dom nature of the weights applied, the results are in general not better than those of the random action policy of the previous section: In [26]: def run_episode(env, weights): state = env.reset() treward = 0 for _ in range(200): s = np.dot(state, weights) a = 0 if s < 0 else 1 state, reward, done, info = env.step(a) treward += reward if done: break return treward In [27]: run_episode(env, weights) Out[27]: 41.0

Therefore, Monte Carlo simulation is applied to test a large number of different weights. The following code simulates a large number of weights, checks them for success or failure, and then chooses the weights that yield success: In [28]: def set_seeds(seed=100): random.seed(seed) np.random.seed(seed) env.seed(seed) In [29]: set_seeds() num_episodes = 1000 In [30]: besttreward = 0 for e in range(1, num_episodes + 1): weights = np.random.rand(4) * 2 - 1 treward = run_episode(env, weights) if treward > besttreward: besttreward = treward bestweights = weights

256

| Chapter 9: Reinforcement Learning

if treward == 200: print(f'SUCCESS | episode={e}') break print(f'UPDATE | episode={e}') UPDATE | episode=1 UPDATE | episode=2 SUCCESS | episode=13 In [31]: weights Out[31]: array([-0.4282,

0.7048,

0.95

,

0.7697])

Random weights. Total reward for these weights. Improvement observed? Replace best total reward. Replace best weights. The CartPole problem is considered solved by an agent if the average total reward over 100 consecutive episodes is 195 or higher. As the following code demonstrates, this is indeed the case for the Monte Carlo agent: In [32]: res = [] for _ in range(100): treward = run_episode(env, weights) res.append(treward) res[:10] Out[32]: [200.0, 200.0, 200.0, 200.0, 200.0, 200.0, 200.0, 200.0, 200.0, 200.0] In [33]: sum(res) / len(res) Out[33]: 200.0

This is, of course, a strong benchmark that other, more sophisticated approaches are up against.

Neural Network Agent The CartPole game can be cast into a classification setting as well. The state of an environment consists of four feature values. The correct action given the feature val‐ ues is the label. By interacting with the environment, a neural network agent can col‐ lect a data set consisting of combinations of feature values and labels. Given this incrementally growing data set, a neural network can be trained to learn the correct action given a state of the environment. The neural network represents the policy in this case. The agent updates the policy based on new experiences.

Neural Network Agent

|

257

First, some imports: In [34]: import tensorflow as tf from keras.layers import Dense, Dropout from keras.models import Sequential from keras.optimizers import Adam, RMSprop from sklearn.metrics import accuracy_score Using TensorFlow backend. In [35]: def set_seeds(seed=100): random.seed(seed) np.random.seed(seed) tf.random.set_seed(seed) env.seed(seed) env.action_space.seed(100)

Second is the NNAgent class that combines the major elements of the agent: the neural network model for the policy, choosing an action given the policy, updating the policy (training the neural network), and the learning process itself over a number of episodes. The agent uses both exploration and exploitation to choose an action. Explo‐ ration refers to a random action, independent of the current policy. Exploitation refers to an action as derived from the current policy. The idea is that some degree of exploration ensures a richer experience and thereby improved learning for the agent: In [36]: class NNAgent: def __init__(self): self.max = 0 self.scores = list() self.memory = list() self.model = self._build_model() def _build_model(self): model = Sequential() model.add(Dense(24, input_dim=4, activation='relu')) model.add(Dense(1, activation='sigmoid')) model.compile(loss='binary_crossentropy', optimizer=RMSprop(lr=0.001)) return model def act(self, state): if random.random() 0.5, 1, 0) return action def train_model(self, state, action): self.model.fit(state, np.array([action,]), epochs=1, verbose=False) def learn(self, episodes):

258

|

Chapter 9: Reinforcement Learning

for e in range(1, episodes + 1): state = env.reset() for _ in range(201): state = np.reshape(state, [1, 4]) action = self.act(state) next_state, reward, done, info = env.step(action) if done: score = _ + 1 self.scores.append(score) self.max = max(score, self.max) print('episode: {:4d}/{} | score: {:3d} | max: {:3d}' .format(e, episodes, score, self.max), end='\r') break self.memory.append((state, action)) self.train_model(state, action) state = next_state

The maximum total reward The DNN classification model for the policy The method to choose an action (exploration and exploitation) The method to update the policy (train the neural network) The method to learn from interacting with the environment The neural network agent does not solve the problem for the configuration shown. The maximum total reward of 200 is not achieved even once: In [37]: set_seeds(100) agent = NNAgent() In [38]: episodes = 500 In [39]: agent.learn(episodes) episode: 500/500 | score: 11 | max: In [40]: sum(agent.scores) / len(agent.scores) Out[40]: 13.682

44

Average total reward over all episodes Something seems to be missing with this approach. One major missing element is the idea of looking beyond the current state and action to be chosen. The approach implemented does not, by any means, take into account that success is only achieved when the agent survives 200 consecutive steps. Simply speaking, the agent avoids tak‐ ing the wrong action but does not learn to win the game. Analyzing the collected history of states (features) and actions (labels) reveals that the neural network reaches an accuracy of around 75%. Neural Network Agent

|

259

However, this does not translate into a winning policy as seen before: In [41]: f = np.array([m[0][0] for f Out[41]: array([[-0.0163, 0.0238, [-0.0158, 0.2195, [-0.0114, 0.0249, ..., [ 0.0603, 0.9682, [ 0.0797, 1.1642, [ 0.103 , 1.3604,

m in agent.memory]) -0.0392, -0.0148], -0.0395, -0.3196], -0.0459, -0.0396], -0.0852, -1.4595], -0.1144, -1.7776], -0.15 , -2.1035]])

In [42]: l = np.array([m[1] for m in agent.memory]) l Out[42]: array([1, 0, 1, ..., 1, 1, 1]) In [43]: accuracy_score(np.where(agent.model.predict(f) > 0.5, 1, 0), l) Out[43]: 0.7525626872733008

Features (states) from all episodes Labels (actions) from all episodes

DQL Agent Q-learning (QL) is an algorithm that takes into account delayed rewards in addition to immediate rewards from an action. The algorithm is due to Watkins (1989) and Watkins and Dayan (1992) and is explained in detail in Sutton and Barto (2018, ch. 6). QL addresses the problem of looking beyond the immediate next reward as encountered with the neural network agent. The algorithm works roughly as follows. There is an action-value policy Q, which assigns a value to every combination of a state and an action. The higher the value is, the better the action from the point of view of the agent will be. If the agent uses the policy Q to choose an action, it selects the action with the highest value. How is the value of an action derived? The value of an action is composed of its direct reward and the discounted value of the optimal action in the next state. The following is the formal expression: Q St, At = Rt + 1 + γ max Q St + 1, a a

Here, St is the state at step (time) t, At is the action taken at state St, Rt + 1 is the direct reward of action At, 0 < γ < 1 is a discount factor, and maxa Q St + 1, a is the maxi‐ mum delayed reward given the optimal action from the current policy Q.

260

|

Chapter 9: Reinforcement Learning

In a simple environment, with only a limited number of possible states, Q can, for example, be represented by a table, listing for every state-action combination the cor‐ responding value. However, in more interesting or complex settings, such as the CartPole environment, the number of states is too large for Q to be written out com‐ prehensively. Therefore, Q is in general understood to be a function. This is where neural networks come into play. In realistic settings and environments, a closed-form solution for the function Q might not exist or might be too hard to derive, say, based on dynamic programming. Therefore, QL algorithms generally tar‐ get approximations only. Neural networks, with their universal approximation capa‐ bilities, are a natural choice to accomplish the approximation of Q. Another critical element of QL is replay. The QL agent replays a number of experien‐ ces (state-action combinations) to update the policy function Q regularly. This can improve the learning considerably. Furthermore, the QL agent presented in the fol‐ lowing—DQLAgent—also alternates between exploration and exploitation during the learning. The alternation is done in a systematic way in that the agent starts with exploration only—in the beginning it could not have learned anything—and slowly but steadily decreases the exploration rate � until it reaches a minimum level:3 In [44]: from collections import deque from keras.optimizers import Adam, RMSprop In [45]: class DQLAgent: def __init__(self, gamma=0.95, hu=24, opt=Adam, lr=0.001, finish=False): self.finish = finish self.epsilon = 1.0 self.epsilon_min = 0.01 self.epsilon_decay = 0.995 self.gamma = gamma self.batch_size = 32 self.max_treward = 0 self.averages = list() self.memory = deque(maxlen=2000) self.osn = env.observation_space.shape[0] self.model = self._build_model(hu, opt, lr) def _build_model(self, hu, opt, lr): model = Sequential() model.add(Dense(hu, input_dim=self.osn, activation='relu')) model.add(Dense(hu, activation='relu')) model.add(Dense(env.action_space.n, activation='linear')) model.compile(loss='mse', optimizer=opt(lr=lr)) return model

3 The implementation is similar to the one found in this blog post.

DQL Agent

|

261

def act(self, state): if random.random() self.epsilon_min: self.epsilon *= self.epsilon_decay def learn(self, episodes): trewards = [] for e in range(1, episodes + 1): state = env.reset() state = np.reshape(state, [1, self.osn]) for _ in range(5000): action = self.act(state) next_state, reward, done, info = env.step(action) next_state = np.reshape(next_state, [1, self.osn]) self.memory.append([state, action, reward, next_state, done]) state = next_state if done: treward = _ + 1 trewards.append(treward) av = sum(trewards[-25:]) / 25 self.averages.append(av) self.max_treward = max(self.max_treward, treward) templ = 'episode: {:4d}/{} | treward: {:4d} | ' templ += 'av: {:6.1f} | max: {:4d}' print(templ.format(e, episodes, treward, av, self.max_treward), end='\r') break if av > 195 and self.finish: break if len(self.memory) > self.batch_size: self.replay() def test(self, episodes): trewards = [] for e in range(1, episodes + 1): state = env.reset()

262

|

Chapter 9: Reinforcement Learning

for _ in range(5001): state = np.reshape(state, [1, self.osn]) action = np.argmax(self.model.predict(state)[0]) next_state, reward, done, info = env.step(action) state = next_state if done: treward = _ + 1 trewards.append(treward) print('episode: {:4d}/{} | treward: {:4d}' .format(e, episodes, treward), end='\r') break return trewards

Initial exploration rate Minimum exploration rate Decay rate for exploration rate Discount factor for delayed reward Batch size for replay deque collection for limited history

Random selection of history batch for replay Q value for state-action pair

Update of the neural network for the new action-value pairs Update of the exploration rate Storing the new data Replay to update the policy based on past experiences How does the QL agent perform? As the code that follows shows, it reaches a winning state for CartPole of a total reward of 200. Figure 9-2 shows the moving average of scores and how it increases over time, although not monotonically. To the contrary, the performance of the agent can significantly decrease at times, as Figure 9-2 shows. Among other things, the exploration that is taking place throughout leads to random actions that might not necessarily lead to good results in terms of total rewards but may lead to beneficial experiences for updating the policy network: In [46]: episodes = 1000 In [47]: set_seeds(100)

DQL Agent

|

263

agent = DQLAgent(finish=True) In [48]: agent.learn(episodes) episode: 400/1000 | treward: 200 | av: 195.4 | max: 200 In [49]: plt.figure(figsize=(10, 6)) x = range(len(agent.averages)) y = np.polyval(np.polyfit(x, agent.averages, deg=3), x) plt.plot(agent.averages, label='moving average') plt.plot(x, y, 'r--', label='trend') plt.xlabel('episodes') plt.ylabel('total reward') plt.legend();

Figure 9-2. Average total rewards of DQLAgent for CartPole Does the QL agent solve the CartPole problem? In this particular case, it does, given the definition of success by OpenAI Gym: In [50]: trewards = agent.test(100) episode: 100/100 | treward: 200 In [51]: sum(trewards) / len(trewards) Out[51]: 200.0

Simple Finance Gym To transfer the QL approach to the financial domain, this section provides a class that mimics an OpenAI Gym environment, but for a financial market as represented by financial time series data. The idea is that, similar to the CartPole environment, four historical prices represent the state of the financial market. An agent can decide, when presented with the state, whether to go long or to go short. In that case, the two

264

|

Chapter 9: Reinforcement Learning

environments are comparable since a state is given by four parameters and an agent can take two different actions. To mimic the OpenAI Gym API, two helper classes are needed—one for the observa‐ tion space, and one for the action space: In [52]: class observation_space: def __init__(self, n): self.shape = (n,) In [53]: class action_space: def __init__(self, n): self.n = n def seed(self, seed): pass def sample(self): return random.randint(0, self.n - 1)

The following Python code defines the Finance class. It retrieves end-of-day histori‐ cal prices for a number of symbols. The major methods of the class are .reset() and .step(). The .step() method checks whether the right action has been taken, defines the reward accordingly, and checks for success or failure. A success is achieved when the agent is able to correctly trade through the whole data set. This can, of course, be defined differently (say, a success is achieved when the agent trades successfully for 1,000 steps). A failure is defined as an accuracy ratio of less than 50% (total rewards divided by total number of steps). However, this is only checked for after a certain number of steps to avoid the high initial variance of this metric: In [54]: class Finance: url = 'http://hilpisch.com/aiif_eikon_eod_data.csv' def __init__(self, symbol, features): self.symbol = symbol self.features = features self.observation_space = observation_space(4) self.osn = self.observation_space.shape[0] self.action_space = action_space(2) self.min_accuracy = 0.475 self._get_data() self._prepare_data() def _get_data(self): self.raw = pd.read_csv(self.url, index_col=0, parse_dates=True).dropna() def _prepare_data(self): self.data = pd.DataFrame(self.raw[self.symbol]) self.data['r'] = np.log(self.data / self.data.shift(1)) self.data.dropna(inplace=True) self.data = (self.data - self.data.mean()) / self.data.std() self.data['d'] = np.where(self.data['r'] > 0, 1, 0) def _get_state(self): return self.data[self.features].iloc[ self.bar - self.osn:self.bar].values

Simple Finance Gym

|

265

def seed(self, seed=None): pass def reset(self): self.treward = 0 self.accuracy = 0 self.bar = self.osn state = self.data[self.features].iloc[ self.bar - self.osn:self.bar] return state.values def step(self, action): correct = action == self.data['d'].iloc[self.bar] reward = 1 if correct else 0 self.treward += reward self.bar += 1 self.accuracy = self.treward / (self.bar - self.osn) if self.bar >= len(self.data): done = True elif reward == 1: done = False elif (self.accuracy < self.min_accuracy and self.bar > self.osn + 10): done = True else: done = False state = self._get_state() info = {} return state, reward, done, info

Defines the minimum accuracy required. Selects the data defining the state of the financial market. Resets the environment to its initial values. Checks whether the agent has chosen the right action (successful trade). Defines the reward the agent receives. Adds the reward to the total reward. Moves the environment one step forward. Calculates the accuracy of successful actions (trades) given all steps (trades). If the agent reaches the end of the data set, success is achieved. If the agent takes the right action, it can move on.

266

|

Chapter 9: Reinforcement Learning

If, after some initial steps, the accuracy drops under the minimum level, the epi‐ sode ends (failure). For the remaining cases, the agent can move on. Instances of the Finance class behave like an environment of the OpenAI Gym. In particular, in this base case, the instance behaves exactly like the CartPole environment: In [55]: env = Finance('EUR=', 'EUR=') In [56]: env.reset() Out[56]: array([1.819 , 1.8579, 1.7749, 1.8579]) In [57]: a = env.action_space.sample() a Out[57]: 0 In [58]: env.step(a) Out[58]: (array([1.8579, 1.7749, 1.8579, 1.947 ]), 0, False, {})

Specifies which symbol and which type of feature (symbol or log return) to be used to define the data representing the state Can the DQLAgent, as developed for the CartPole game, learn to trade in a financial market? Yes, it can, as the following code illustrates. However, although the agent improves its trading skill (on average) over the training episodes, the results are not too impressive (see Figure 9-3): In [59]: set_seeds(100) agent = DQLAgent(gamma=0.5, opt=RMSprop) In [60]: episodes = 1000 In [61]: agent.learn(episodes) episode: 1000/1000 | treward: 2511 | av: 1012.7 | max: 2511 In [62]: agent.test(3) episode: 3/3 | treward: 2511 Out[62]: [2511, 2511, 2511] In [63]: plt.figure(figsize=(10, 6)) x = range(len(agent.averages)) y = np.polyval(np.polyfit(x, agent.averages, deg=3), x) plt.plot(agent.averages, label='moving average') plt.plot(x, y, 'r--', label='regression') plt.xlabel('episodes') plt.ylabel('total reward') plt.legend();

Simple Finance Gym

|

267

Figure 9-3. Average total rewards of DQLAgent for Finance

General RL Agents This section provides a class for a financial market environment that mimics the API of an OpenAI Gym environment. It also applies, without any changes to the agent itself, the QL agent to the new financial market environment. Although the performance of the agent in this new environment might not be impressive, it illus‐ trates that the approach of RL, as introduced in this chapter, is rather general. RL agents can in general learn from different envi‐ ronments they interact with. This explains to some extent why AlphaZero from DeepMind is able to master not only the game of Go but also chess and shogi, as discussed in Chapter 2.

Better Finance Gym The idea in the previous section is to develop a simple class that allows RL within a financial market setting. The major goal in that section is to replicate the API of an OpenAI Gym environment. However, there is no need to restrict such an environ‐ ment to a single type of feature to describe the state of the financial market nor to use only four lags. This section introduces an improved Finance class that allows for multiple features, a flexible number of lags, and specific start and end points for the base data set used. This, among other things, allows the use of one part of the data set for learning and another one for validation or testing. The Python code presented in the following also allows the use of leverage. This might be helpful when intraday data is considered with relatively small absolute returns:

268

|

Chapter 9: Reinforcement Learning

In [64]: class Finance: url = 'http://hilpisch.com/aiif_eikon_eod_data.csv' def __init__(self, symbol, features, window, lags, leverage=1, min_performance=0.85, start=0, end=None, mu=None, std=None): self.symbol = symbol self.features = features self.n_features = len(features) self.window = window self.lags = lags self.leverage = leverage self.min_performance = min_performance self.start = start self.end = end self.mu = mu self.std = std self.observation_space = observation_space(self.lags) self.action_space = action_space(2) self._get_data() self._prepare_data() def _get_data(self): self.raw = pd.read_csv(self.url, index_col=0, parse_dates=True).dropna() def _prepare_data(self): self.data = pd.DataFrame(self.raw[self.symbol]) self.data = self.data.iloc[self.start:] self.data['r'] = np.log(self.data / self.data.shift(1)) self.data.dropna(inplace=True) self.data['s'] = self.data[self.symbol].rolling( self.window).mean() self.data['m'] = self.data['r'].rolling(self.window).mean() self.data['v'] = self.data['r'].rolling(self.window).std() self.data.dropna(inplace=True) if self.mu is None: self.mu = self.data.mean() self.std = self.data.std() self.data_ = (self.data - self.mu) / self.std self.data_['d'] = np.where(self.data['r'] > 0, 1, 0) self.data_['d'] = self.data_['d'].astype(int) if self.end is not None: self.data = self.data.iloc[:self.end - self.start] self.data_ = self.data_.iloc[:self.end - self.start] def _get_state(self): return self.data_[self.features].iloc[self.bar self.lags:self.bar] def seed(self, seed): random.seed(seed) np.random.seed(seed) def reset(self): self.treward = 0 self.accuracy = 0 self.performance = 1

Better Finance Gym

|

269

self.bar = self.lags state = self.data_[self.features].iloc[self.barself.lags:self.bar] return state.values def step(self, action): correct = action == self.data_['d'].iloc[self.bar] ret = self.data['r'].iloc[self.bar] * self.leverage reward_1 = 1 if correct else 0 reward_2 = abs(ret) if correct else -abs(ret) factor = 1 if correct else -1 self.treward += reward_1 self.bar += 1 self.accuracy = self.treward / (self.bar - self.lags) self.performance *= math.exp(reward_2) if self.bar >= len(self.data): done = True elif reward_1 == 1: done = False elif (self.performance < self.min_performance and self.bar > self.lags + 5): done = True else: done = False state = self._get_state() info = {} return state.values, reward_1 + reward_2 * 5, done, info

The features to define the state The number of lags to be used The minimum gross performance required Additional financial features (simple moving average, momentum, rolling volatility) Gaussian normalization of the data The leveraged return for the step The return-based reward for the step The gross performance after the step The new Finance class gives more flexibility for the modeling of the financial market environment. The following code shows an example for two features and five lags: In [65]: env = Finance('EUR=', ['EUR=', 'r'], 10, 5) In [66]: a = env.action_space.sample()

270

|

Chapter 9: Reinforcement Learning

a Out[66]: 0 In [67]: env.reset() Out[67]: array([[ 1.7721, -1.0214], [ 1.5973, -2.4432], [ 1.5876, -0.1208], [ 1.6292, 0.6083], [ 1.6408, 0.1807]]) In [68]: env.step(a) Out[68]: (array([[ 1.5973, -2.4432], [ 1.5876, -0.1208], [ 1.6292, 0.6083], [ 1.6408, 0.1807], [ 1.5725, -0.9502]]), 1.0272827803740798, False, {})

Different Types of Environments and Data It is important to notice that there is a fundamental difference between the CartPole environment and the two versions of the Finance environment. In the CartPole envi‐ ronment, no data is available up front. Only an initial state is chosen with some degree of randomness. Given this state and the action taken by an agent, determinis‐ tic transformations are applied to generate new states (data). This is possible since a physical system is simulated that follows physical laws. The Finance environment, on the other hand, starts with real, historical market data and only presents the available data to the agent in similar fashion as the CartPole environment (that is, step by step and state by state). In this case, the action of the agent does not really influence the environment; the environment instead evolves deterministically, and the agent learns how to behave optimally—trade profitably—in that environment. In that sense, the Finance environment is more comparable, say, to the problem of finding the fastest way through a labyrinth. In such a case, the data representing the labyrinth is given up front and the agent is only presented with the relevant sub-set of the data (the current state) as it moves around the labyrinth.

FQL Agent Relying on the new Finance environment, this section improves on the simple DQL agent to improve the performance in the financial market context. The FQLAgent class is able to handle multiple features and a flexible number of lags. It also

FQL Agent

|

271

distinguishes the learning environment (learn_env) from the validation environment (valid_env). This allows one to gain a more realistic picture of the out-of-sample performance of the agent during training. The basic structure of the class and the RL/QL learning approach is the same for both the DQLAgent class and the FQLAgent class: In [69]: class FQLAgent: def __init__(self, hidden_units, learning_rate, learn_env, valid_env): self.learn_env = learn_env self.valid_env = valid_env self.epsilon = 1.0 self.epsilon_min = 0.1 self.epsilon_decay = 0.98 self.learning_rate = learning_rate self.gamma = 0.95 self.batch_size = 128 self.max_treward = 0 self.trewards = list() self.averages = list() self.performances = list() self.aperformances = list() self.vperformances = list() self.memory = deque(maxlen=2000) self.model = self._build_model(hidden_units, learning_rate) def _build_model(self, hu, lr): model = Sequential() model.add(Dense(hu, input_shape=( self.learn_env.lags, self.learn_env.n_features), activation='relu')) model.add(Dropout(0.3, seed=100)) model.add(Dense(hu, activation='relu')) model.add(Dropout(0.3, seed=100)) model.add(Dense(2, activation='linear')) model.compile( loss='mse', optimizer=RMSprop(lr=lr) ) return model def act(self, state): if random.random() self.epsilon_min: self.epsilon *= self.epsilon_decay def learn(self, episodes): for e in range(1, episodes + 1): state = self.learn_env.reset() state = np.reshape(state, [1, self.learn_env.lags, self.learn_env.n_features]) for _ in range(10000): action = self.act(state) next_state, reward, done, info = \ self.learn_env.step(action) next_state = np.reshape(next_state, [1, self.learn_env.lags, self.learn_env.n_features]) self.memory.append([state, action, reward, next_state, done]) state = next_state if done: treward = _ + 1 self.trewards.append(treward) av = sum(self.trewards[-25:]) / 25 perf = self.learn_env.performance self.averages.append(av) self.performances.append(perf) self.aperformances.append( sum(self.performances[-25:]) / 25) self.max_treward = max(self.max_treward, treward) templ = 'episode: {:2d}/{} | treward: {:4d} | ' templ += 'perf: {:5.3f} | av: {:5.1f} | max: {:4d}' print(templ.format(e, episodes, treward, perf, av, self.max_treward), end='\r') break self.validate(e, episodes) if len(self.memory) > self.batch_size: self.replay() def validate(self, e, episodes): state = self.valid_env.reset() state = np.reshape(state, [1, self.valid_env.lags, self.valid_env.n_features]) for _ in range(10000): action = np.argmax(self.model.predict(state)[0, 0]) next_state, reward, done, info = self.valid_env.step(action) state = np.reshape(next_state, [1, self.valid_env.lags, self.valid_env.n_features]) if done: treward = _ + 1

FQL Agent

|

273

perf = self.valid_env.performance self.vperformances.append(perf) if e % 20 == 0: templ = 71 * '=' templ += '\nepisode: {:2d}/{} | VALIDATION | ' templ += 'treward: {:4d} | perf: {:5.3f} | ' templ += 'eps: {:.2f}\n' templ += 71 * '=' print(templ.format(e, episodes, treward, perf, self.epsilon)) break

The following Python code shows that the performance of the FQLAgent is substan‐ tially better than that of the simple DQLAgent that solves the CartPole problem. This trading bot seems to learn about trading rather consistently through interacting with the financial market environment (see Figure 9-4): In [70]: symbol = 'EUR=' features = [symbol, 'r', 's', 'm', 'v'] In [71]: a = 0 b = 2000 c = 500 In [72]: learn_env = Finance(symbol, features, window=10, lags=6, leverage=1, min_performance=0.85, start=a, end=a + b, mu=None, std=None) In [73]: learn_env.data.info()

DatetimeIndex: 2000 entries, 2010-01-19 to 2017-12-26 Data columns (total 5 columns): # Column Non-Null Count Dtype --- ------ -------------- ----0 EUR= 2000 non-null float64 1 r 2000 non-null float64 2 s 2000 non-null float64 3 m 2000 non-null float64 4 v 2000 non-null float64 dtypes: float64(5) memory usage: 93.8 KB In [74]: valid_env = Finance(symbol, features, window=learn_env.window, lags=learn_env.lags, leverage=learn_env.leverage, min_performance=learn_env.min_performance, start=a + b, end=a + b + c, mu=learn_env.mu, std=learn_env.std) In [75]: valid_env.data.info()

DatetimeIndex: 500 entries, 2017-12-27 to 2019-12-20 Data columns (total 5 columns):

274

|

Chapter 9: Reinforcement Learning

# Column Non-Null Count Dtype --- ------ -------------- ----0 EUR= 500 non-null float64 1 r 500 non-null float64 2 s 500 non-null float64 3 m 500 non-null float64 4 v 500 non-null float64 dtypes: float64(5) memory usage: 23.4 KB In [76]: set_seeds(100) agent = FQLAgent(24, 0.0001, learn_env, valid_env) In [77]: episodes = 61 In [78]: agent.learn(episodes) ======================================================================= episode: 20/61 | VALIDATION | treward: 494 | perf: 1.169 | eps: 0.68 ======================================================================= ======================================================================= episode: 40/61 | VALIDATION | treward: 494 | perf: 1.111 | eps: 0.45 ======================================================================= ======================================================================= episode: 60/61 | VALIDATION | treward: 494 | perf: 1.089 | eps: 0.30 ======================================================================= episode: 61/61 | treward: 1994 | perf: 1.268 | av: 1615.1 | max: 1994 In [79]: agent.epsilon Out[79]: 0.291602079838278 In [80]: plt.figure(figsize=(10, 6)) x = range(1, len(agent.averages) + 1) y = np.polyval(np.polyfit(x, agent.averages, deg=3), x) plt.plot(agent.averages, label='moving average') plt.plot(x, y, 'r--', label='regression') plt.xlabel('episodes') plt.ylabel('total reward') plt.legend();

FQL Agent

|

275

Figure 9-4. Average total rewards of FQLAgent for Finance An interesting picture also arises for the training and validation performances, as shown in Figure 9-5. The training performance shows a high variance, which is due, for example, to the exploration that is going on in addition to the exploitation of the currently optimal policy. In comparison, the validation performance has a much lower variance because it only relies on the exploitation of the currently optimal policy: In [81]: plt.figure(figsize=(10, 6)) x = range(1, len(agent.performances) + 1) y = np.polyval(np.polyfit(x, agent.performances, deg=3), x) y_ = np.polyval(np.polyfit(x, agent.vperformances, deg=3), x) plt.plot(agent.performances[:], label='training') plt.plot(agent.vperformances[:], label='validation') plt.plot(x, y, 'r--', label='regression (train)') plt.plot(x, y_, 'r-.', label='regression (valid)') plt.xlabel('episodes') plt.ylabel('gross performance') plt.legend();

276

| Chapter 9: Reinforcement Learning

Figure 9-5. Training and validation performance of the FQLAgent per episode

Conclusions This chapter discusses reinforcement learning as one of the most successful algorithm classes that AI has to offer. Most of the advances and success stories discussed in Chapter 2 have their origin in improvements in the field of RL. In this context, neural networks are not rendered useless. To the contrary, they play an important role in approximating the optimal action policy, usually in the form of a policy Q that, given a certain state, assigns each action a value. The higher the value is, the better the action will be, taking into account both immediate and delayed rewards. The inclusion of delayed rewards, of course, is relevant in many important contexts. In a gaming context, with multiple actions available in general, it is optimal to choose the one that promises the highest total reward—and probably not just the highest immediate reward. The final total score is what is to be maximized. The same holds true in a financial context. The long-term performance is in general the appropriate goal for trading and investing, not a quick short-term profit that might come at an increased risk of going bankrupt. The examples in this chapter also demonstrate that the RL approach is rather flexible and general in that it can be applied to different settings equally well. The DQL agent that solves the CartPole problem can also learn how to trade in a financial market, although not too well. Based on improvements of the Finance environment and the FQL agent, the FQL trading bot shows a respectable performance both in-sample (on the training data) and out-of-sample (on the validation data).

Conclusions

|

277

References Books and papers cited in this chapter: Sutton, Richard S. and Andrew G. Barto. 2018. Reinforcement Learning: An Introduc‐ tion. Cambridge and London: MIT Press. Watkins, Christopher. 1989. Learning from Delayed Rewards. Ph.D. thesis, University of Cambridge. Watkins, Christopher and Peter Dayan. 1992. “Q-Learning.” Machine Learning 8 (May): 279-282.

278

|

Chapter 9: Reinforcement Learning

PART IV

Algorithmic Trading

Success means making profits and avoiding losses. —Martin Zweig

Part III is concerned with the discovery of statistical inefficiencies in financial mar‐ kets by the use of deep learning and reinforcement learning techniques. This part, by contrast, is concerned with identifying and exploiting economic inefficiencies for which statistical inefficiencies are a prerequisite in general. The tool of choice for exploiting economic inefficiencies is algorithmic trading, that is, the automated execu‐ tion of trading strategies based on predictions generated by a trading bot. Table IV-1 compares in a simplified manner the problem of training and deploying a trading bot with the one of building and deploying a self-driving car. Table IV-1. Self-driving cars compared to trading bots Step Training

Self-Driving Car Training AI in virtual and recorded environments

Trading Bot Training AI with simulated and real historical data

Risk management

Adding rules to avoid collisions, crashes, and so on

Adding rules to avoid large losses, to take profits early, and so on

Deployment

Combining AI with car hardware, deploying the car on the street, and monitoring

Combining AI with trading platform, deploying the trading bot for real trading, and monitoring

This part consists of three chapters that are structured along the three steps, as illus‐ trated in Table IV-1, to exploit economic inefficiencies through a trading bot—start‐ ing with the vectorized backtesting of trading strategies, covering the analysis of risk

management measures through event-based backtesting, and discussing technical details in the context of strategy execution and deployment: • Chapter 10 is about the vectorized backtesting of algorithmic trading strategies, such as those based on a DNN for market prediction. This approach is both effi‐ cient and insightful with regard to a first judgment of the economic potential of a trading strategy. It also allows one to assess the impact of transaction costs on economic performance. • Chapter 11 covers central aspects of managing the risk of algorithmic trading strategies, such as the use of stop loss orders or take profit orders. In addition to vectorized backtesting, this chapter introduces event-based backtesting as a more flexible approach to judge the economic potential of a trading strategy. • Chapter 12 is primarily about the execution of trading strategies. Topics are the retrieval of historical data, the training of a trading bot based on this data, the streaming of real-time data, and the placement of orders. It introduces Oanda and its API as a trading platform well suited to algorithmic trading. It also covers fundamental aspects of deploying AI-powered algorithmic trading strategies in automatic fashion.

Algorithmic Trading Strategies Algorithmic trading is a vast field and encompasses different types of trading strategies. Some, for example, try to minimize the mar‐ ket impact during the execution of large orders (liquidity algorithms). Others try to replicate the payoff of derivatives instru‐ ments as closely as possible (dynamic hedging/replication). These examples illustrate that not all algorithmic trading strategies have the goal of exploiting economic inefficiencies. For the purposes of this book, the focus on algorithmic trading strategies that result from predictions made by a trading bot (for example, in the form of a DNN agent or an RL agent) seems appropriate and useful.

CHAPTER 10

Vectorized Backtesting

Tesla’s chief executive and serial technology entrepreneur, Elon Musk, has said his company’s cars will be able to be summoned and drive autonomously across the US to pick up their owners within the next two years. —Samuel Gibbs (2016) Big money is made in the stock market by being on the right side of the major moves. —Martin Zweig

The term vectorized backtesting refers to a technical approach to backtesting algorith‐ mic trading strategies, such as those based on a dense neural network (DNN) for market prediction. The books by Hilpisch (2018, ch. 15; 2020, ch. 4) cover vectorized backtesting based on a number of concrete examples. Vectorized in this context refers to a programming paradigm that relies heavily or even exclusively on vectorized code (that is, code without any looping on the Python level). Vectorization of code is good practice with such packages such as Numpy or pandas in general and has been used intensively in previous chapters as well. The benefits of vectorized code are more concise and easy-to-read code, as well as faster execution in many important scenar‐ ios. On the other hand, it might not be as flexible in backtesting trading strategies as, for example, event-based backtesting, which is introduced and used in Chapter 11. Having a good AI-powered predictor available that beats a simple baseline predictor is important but is generally not enough to generate alpha (that is, above-market returns, possibly adjusted for risk). For example, it is also important for a predictionbased trading strategy to predict the large market movements correctly and not just the majority of the (potentially pretty small) market movements. Vectorized backtest‐ ing is an easy and fast way of figuring out the economic potential of a trading strategy.

281

Compared to autonomous vehicles (AVs), vectorized backtesting is like testing the AI of AVs in virtual environments just to see how it performs “in general” in a risk-free environment. However, for the AI of an AV it is not only important to perform well on average, but it is also of paramount importance to see how it masters critical or even extreme situations. Such an AI is supposed to cause “zero casualties” on average, not 0.1 or 0.5. For a financial AI, it is similarly—even if not equally—important to get the large market movements correct. Whereas this chapter focuses on the pure per‐ formance of financial AI agents (trading bots), Chapter 11 goes deeper into risk assessment and the backtesting of standard risk measures. “Backtesting an SMA-Based Strategy” on page 282 introduces vectorized backtesting based on a simple example using simple moving averages as technical indicators and end-of-day (EOD) data. This allows for insightful visualizations and an easier under‐ standing of the approach when getting started. “Backtesting a Daily DNN-Based Strategy” on page 289 trains a DNN based on EOD data and backtests the resulting prediction-based strategy for its economic performance. “Backtesting an Intraday DNN-Based Strategy” on page 295 then does the same with intraday data. In all examples, proportional transaction costs are included in the form of assumed bid-ask spreads.

Backtesting an SMA-Based Strategy This section introduces vectorized backtesting based on a classical trading strategy that uses simple moving averages (SMAs) as technical indicators. The following code realizes the necessary imports and configurations and retrieves EOD data for the EUR/USD currency pair: In [1]: import os import math import numpy as np import pandas as pd from pylab import plt, mpl plt.style.use('seaborn') mpl.rcParams['savefig.dpi'] = 300 mpl.rcParams['font.family'] = 'serif' pd.set_option('mode.chained_assignment', None) pd.set_option('display.float_format', '{:.4f}'.format) np.set_printoptions(suppress=True, precision=4) os.environ['PYTHONHASHSEED'] = '0' In [2]: url = 'http://hilpisch.com/aiif_eikon_eod_data.csv' In [3]: symbol = 'EUR=' In [4]: data = pd.DataFrame(pd.read_csv(url, index_col=0, parse_dates=True).dropna()[symbol])

282

|

Chapter 10: Vectorized Backtesting

In [5]: data.info()

DatetimeIndex: 2516 entries, 2010-01-04 to 2019-12-31 Data columns (total 1 columns): # Column Non-Null Count Dtype --- ------ -------------- ----0 EUR= 2516 non-null float64 dtypes: float64(1) memory usage: 39.3 KB

Retrieves EOD data for EUR/USD The idea of the strategy is the following. Calculate a shorter SMA1, say for 42 days, and a longer SMA2, say for 258 days. Whenever SMA1 is above SMA2, go long on the financial instrument. Whenever SMA1 is below SMA2, go short on the financial instru‐ ment. Because the example is based on EUR/USD, going long or short is easily accomplished. The following Python code calculates in vectorized fashion the SMA values and visu‐ alizes the resulting time series alongside the original time series (see Figure 10-1): In [6]: data['SMA1'] = data[symbol].rolling(42).mean() In [7]: data['SMA2'] = data[symbol].rolling(258).mean() In [8]: data.plot(figsize=(10, 6));

Calculates the shorter SMA1 Calculates the longer SMA2 Visualizes the three time series Equipped with the SMA time series data, the resulting positions can, again in vector‐ ized fashion, be derived. Note the shift of the resulting position time series by one day to avoid foresight bias in the data. This shift is necessary since the calculation of the SMAs includes the closing values from the same day. Therefore, the position derived from the SMA values from one day needs to be applied to the next day for the whole time series.

Backtesting an SMA-Based Strategy

|

283

Figure 10-1. Time series data for EUR/USD and SMAs Figure 10-2 visualizes the resulting positions as an overlay to the other time series: In [9]: data.dropna(inplace=True) In [10]: data['p'] = np.where(data['SMA1'] > data['SMA2'], 1, -1) In [11]: data['p'] = data['p'].shift(1) In [12]: data.dropna(inplace=True) In [13]: data.plot(figsize=(10, 6), secondary_y='p');

Deletes rows containing NaN values Derives the position values based on same-day SMA values Shifts the position values by one day to avoid foresight bias Visualizes the position values as derived from the SMAs

284

|

Chapter 10: Vectorized Backtesting

Figure 10-2. Time series data for EUR/USD, SMAs, and resulting positions One crucial step is missing: the combination of the positions with the returns of the financial instrument. Since positions are conveniently represented by a +1 for a long position and a -1 for a short position, this step boils down to multiplying two col‐ umns of the DataFrame object—in vectorized fashion again. The SMA-based trading strategy outperforms the passive benchmark investment by a considerable margin, as Figure 10-3 illustrates: In [14]: data['r'] = np.log(data[symbol] / data[symbol].shift(1)) In [15]: data.dropna(inplace=True) In [16]: data['s'] = data['p'] * data['r'] In [17]: data[['r', 's']].sum().apply(np.exp) Out[17]: r 0.8640 s 1.3773 dtype: float64 In [18]: data[['r', 's']].sum().apply(np.exp) - 1 Out[18]: r -0.1360 s 0.3773 dtype: float64 In [19]: data[['r', 's']].cumsum().apply(np.exp).plot(figsize=(10, 6));

Calculates the log returns Calculates the strategy returns Calculates the gross performances Backtesting an SMA-Based Strategy

|

285

Calculates the net performances Visualizes the gross performances over time

Figure 10-3. Gross performance of passive benchmark investment and SMA strategy So far, the performance figures are not considering transaction costs. These are, of course, a crucial element when judging the economic potential of a trading strategy. In the current setup, proportional transaction costs can be easily included in the cal‐ culations. The idea is to determine when a trade takes place and to reduce the perfor‐ mance of the trading strategy by a certain value to account for the relevant bid-ask spread. As the following calculations show, and as is obvious from Figure 10-2, the trading strategy does not change positions too often. Therefore, in order to have some meaningful effects of transaction costs, they are assumed to be quite a bit higher than typically seen for EUR/USD. The net effect of subtracting transaction costs is a few percentage points under the given assumptions (see Figure 10-4): In [20]: sum(data['p'].diff() != 0) + 2 Out[20]: 10 In [21]: pc = 0.005 In [22]: data['s_'] = np.where(data['p'].diff() != 0, data['s'] - pc, data['s']) In [23]: data['s_'].iloc[0] -= pc In [24]: data['s_'].iloc[-1] -= pc In [25]: data[['r', 's', 's_']][data['p'].diff() != 0] Out[25]: r s s_

286

|

Chapter 10: Vectorized Backtesting

Date 2011-01-12 2011-10-10 2012-11-07 2014-07-24 2016-03-16 2016-11-10 2017-06-05 2018-06-15

0.0123 0.0198 -0.0034 -0.0001 0.0102 -0.0018 -0.0025 0.0035

0.0123 -0.0198 -0.0034 0.0001 0.0102 0.0018 -0.0025 -0.0035

0.0023 -0.0248 -0.0084 -0.0049 0.0052 -0.0032 -0.0075 -0.0085

In [26]: data[['r', 's', 's_']].sum().apply(np.exp) Out[26]: r 0.8640 s 1.3773 s_ 1.3102 dtype: float64 In [27]: data[['r', 's', 's_']].sum().apply(np.exp) - 1 Out[27]: r -0.1360 s 0.3773 s_ 0.3102 dtype: float64 In [28]: data[['r', 's', 's_']].cumsum().apply(np.exp).plot(figsize=(10, 6));

Calculates the number of trades, including entry and exit trade Fixes the proportional transaction costs (deliberately set quite high) Adjusts the strategy performance for the transaction costs Adjusts the strategy performance for the entry trade Adjusts the strategy performance for the exit trade Shows the adjusted performance values for the regular trades

Backtesting an SMA-Based Strategy

|

287

Figure 10-4. Gross performance of the SMA strategy before and after transaction costs What about the resulting risk of the trading strategy? For a trading strategy that is based on directional predictions and that takes long or short positions only, the risk, expressed as the volatility (standard deviation of the log returns), is exactly the same as for the passive benchmark investment: In [29]: data[['r', 's', 's_']].std() Out[29]: r 0.0054 s 0.0054 s_ 0.0054 dtype: float64 In [30]: data[['r', 's', 's_']].std() * math.sqrt(252) Out[30]: r 0.0853 s 0.0853 s_ 0.0855 dtype: float64

Daily volatility Annualized volatility

Vectorized Backtesting Vectorized backtesting is a powerful and efficient approach to backtesting the “pure” performance of a prediction-based trading strategy. It can also accommodate proportional transaction costs, for instance. However, it is not well suited to including typical risk management measures, such as (trailing) stop loss orders or take profit orders. This is addressed in Chapter 11.

288

|

Chapter 10: Vectorized Backtesting

Backtesting a Daily DNN-Based Strategy The previous section lays out the blueprint for vectorized backtesting on the basis of a simple, easy-to-visualize trading strategy. The same blueprint can be applied, for example, to DNN-based trading strategies with minimal technical adjustments. The following trains a Keras DNN model, as discussed in Chapter 7. The data that is used is the same as in the previous example. However, as in Chapter 7, different features and lags thereof need to be added to the DataFrame object: In [31]: data = pd.DataFrame(pd.read_csv(url, index_col=0, parse_dates=True).dropna()[symbol]) In [32]: data.info()

DatetimeIndex: 2516 entries, 2010-01-04 to 2019-12-31 Data columns (total 1 columns): # Column Non-Null Count Dtype --- ------ -------------- ----0 EUR= 2516 non-null float64 dtypes: float64(1) memory usage: 39.3 KB In [33]: lags = 5 In [34]: def add_lags(data, symbol, lags, window=20): cols = [] df = data.copy() df.dropna(inplace=True) df['r'] = np.log(df / df.shift(1)) df['sma'] = df[symbol].rolling(window).mean() df['min'] = df[symbol].rolling(window).min() df['max'] = df[symbol].rolling(window).max() df['mom'] = df['r'].rolling(window).mean() df['vol'] = df['r'].rolling(window).std() df.dropna(inplace=True) df['d'] = np.where(df['r'] > 0, 1, 0) features = [symbol, 'r', 'd', 'sma', 'min', 'max', 'mom', 'vol'] for f in features: for lag in range(1, lags + 1): col = f'{f}_lag_{lag}' df[col] = df[f].shift(lag) cols.append(col) df.dropna(inplace=True) return df, cols In [35]: data, cols = add_lags(data, symbol, lags, window=20)

Backtesting a Daily DNN-Based Strategy

|

289

The following Python code accomplishes additional imports and defines the set_seeds() and create_model() functions: In [36]: import random import tensorflow as tf from keras.layers import Dense, Dropout from keras.models import Sequential from keras.regularizers import l1 from keras.optimizers import Adam from sklearn.metrics import accuracy_score Using TensorFlow backend. In [37]: def set_seeds(seed=100): random.seed(seed) np.random.seed(seed) tf.random.set_seed(seed) set_seeds() In [38]: optimizer = Adam(learning_rate=0.0001) In [39]: def create_model(hl=2, hu=128, dropout=False, rate=0.3, regularize=False, reg=l1(0.0005), optimizer=optimizer, input_dim=len(cols)): if not regularize: reg = None model = Sequential() model.add(Dense(hu, input_dim=input_dim, activity_regularizer=reg, activation='relu')) if dropout: model.add(Dropout(rate, seed=100)) for _ in range(hl): model.add(Dense(hu, activation='relu', activity_regularizer=reg)) if dropout: model.add(Dropout(rate, seed=100)) model.add(Dense(1, activation='sigmoid')) model.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy']) return model

Based on a sequential train-test split of the historical data, the following Python code first trains the DNN model based on normalized features data: In [40]: split = '2018-01-01' In [41]: train = data.loc[:split].copy() In [42]: np.bincount(train['d']) Out[42]: array([ 982, 1006])

290

|

Chapter 10: Vectorized Backtesting

In [43]: mu, std = train.mean(), train.std() In [44]: train_ = (train - mu) / std In [45]: set_seeds() model = create_model(hl=2, hu=64) In [46]: %%time model.fit(train_[cols], train['d'], epochs=20, verbose=False, validation_split=0.2, shuffle=False) CPU times: user 2.93 s, sys: 574 ms, total: 3.5 s Wall time: 1.93 s Out[46]: In [47]: model.evaluate(train_[cols], train['d']) 1988/1988 [==============================] - 0s 17us/step Out[47]: [0.6745863538872549, 0.5925553441047668]

Splits the data into training and test data Shows the frequency of the labels classes Normalizes the training features data Creates the DNN model Trains the DNN model on the training data Evaluates the performance of the model on the training data So far, this basically repeats the core approach of Chapter 7. Vectorized backtesting can now be applied to judge the economic performance of the DNN-based trading strategy in-sample based on the model’s predictions (see Figure 10-5). In this context, an upward prediction is naturally interpreted as a long position and a downward pre‐ diction as a short position: In [48]: train['p'] = np.where(model.predict(train_[cols]) > 0.5, 1, 0) In [49]: train['p'] = np.where(train['p'] == 1, 1, -1) In [50]: train['p'].value_counts() Out[50]: -1 1098 1 890 Name: p, dtype: int64 In [51]: train['s'] = train['p'] * train['r']

Backtesting a Daily DNN-Based Strategy

|

291

In [52]: train[['r', 's']].sum().apply(np.exp) Out[52]: r 0.8787 s 5.0766 dtype: float64 In [53]: train[['r', 's']].sum().apply(np.exp) Out[53]: r -0.1213 s 4.0766 dtype: float64

- 1

In [54]: train[['r', 's']].cumsum().apply(np.exp).plot(figsize=(10, 6));

Generates the binary predictions Translates the predictions into position values Shows the number of long and short positions Calculates the strategy performance values Calculates the gross and net performances (in-sample) Visualizes the gross performances over time (in-sample)

Figure 10-5. Gross performance of the passive benchmark investment and the daily DNN strategy (in-sample)

292

|

Chapter 10: Vectorized Backtesting

Next is the same sequence of calculations for the test data set. Whereas the out-performance in-sample is significant, the numbers out-of-sample are not as impressive but are still convincing (see Figure 10-6): In [55]: test = data.loc[split:].copy() In [56]: test_ = (test - mu) / std In [57]: model.evaluate(test_[cols], test['d']) 503/503 [==============================] - 0s 17us/step Out[57]: [0.6933823573897421, 0.5407554507255554] In [58]: test['p'] = np.where(model.predict(test_[cols]) > 0.5, 1, -1) In [59]: test['p'].value_counts() Out[59]: -1 406 1 97 Name: p, dtype: int64 In [60]: test['s'] = test['p'] * test['r'] In [61]: test[['r', 's']].sum().apply(np.exp) Out[61]: r 0.9345 s 1.2431 dtype: float64 In [62]: test[['r', 's']].sum().apply(np.exp) - 1 Out[62]: r -0.0655 s 0.2431 dtype: float64 In [63]: test[['r', 's']].cumsum().apply(np.exp).plot(figsize=(10, 6));

Generates the test data sub-set Normalizes the test data Evaluates the model performance on the test data The DNN-based trading strategy leads to a larger number of trades as compared to the SMA-based strategy. This makes the inclusion of transaction costs an even more important aspect when judging the economic performance.

Backtesting a Daily DNN-Based Strategy

|

293

Figure 10-6. Gross performance of the passive benchmark investment and the daily DNN strategy (out-of-sample) The following code assumes now realistic bid-ask spreads for EUR/USD on the level of 1.2 pips (that is, 0.00012 in terms of currency units).1 To simplify the calculations, an average value for the proportional transaction costs pc is calculated based on the average closing price for EUR/USD (see Figure 10-7): In [64]: sum(test['p'].diff() != 0) Out[64]: 147 In [65]: spread = 0.00012 pc = spread / data[symbol].mean() print(f'{pc:.6f}') 0.000098 In [66]: test['s_'] = np.where(test['p'].diff() != 0, test['s'] - pc, test['s']) In [67]: test['s_'].iloc[0] -= pc In [68]: test['s_'].iloc[-1] -= pc In [69]: test[['r', 's', 's_']].sum().apply(np.exp) Out[69]: r 0.9345 s 1.2431 s_ 1.2252 dtype: float64

1 This, for example, is a typical spread that Oanda offers retail traders.

294

| Chapter 10: Vectorized Backtesting

In [70]: test[['r', 's', 's_']].sum().apply(np.exp) - 1 Out[70]: r -0.0655 s 0.2431 s_ 0.2252 dtype: float64 In [71]: test[['r', 's', 's_']].cumsum().apply(np.exp).plot(figsize=(10, 6));

Fixes the average bid-ask spread Calculates the average proportional transaction costs

Figure 10-7. Gross performance of the daily DNN strategy before and after transaction costs (out-of-sample) The DNN-based trading strategy seems promising both before and after typical transaction costs. However, would a similar strategy be economically viable intraday as well, when even more trades are observed? The next section analyzes a DNN-based intraday strategy.

Backtesting an Intraday DNN-Based Strategy To train and backtest a DNN model on intraday data, another data set is required: In [72]: url = 'http://hilpisch.com/aiif_eikon_id_eur_usd.csv' In [73]: symbol = 'EUR=' In [74]: data = pd.DataFrame(pd.read_csv(url, index_col=0, parse_dates=True).dropna()['CLOSE']) data.columns = [symbol]

Backtesting an Intraday DNN-Based Strategy

|

295

In [75]: data = data.resample('5min', label='right').last().ffill() In [76]: data.info()

DatetimeIndex: 26486 entries, 2019-10-01 00:05:00 to 2019-12-31 23:10:00 Freq: 5T Data columns (total 1 columns): # Column Non-Null Count Dtype --- ------ -------------- ----0 EUR= 26486 non-null float64 dtypes: float64(1) memory usage: 413.8 KB In [77]: lags = 5 In [78]: data, cols = add_lags(data, symbol, lags, window=20)

Retrieves intraday data for EUR/USD and picks the closing prices Resamples the data to five-minute bars The procedure of the previous section can now be repeated with the new data set. First, train the DNN model: In [79]: split = int(len(data) * 0.85) In [80]: train = data.iloc[:split].copy() In [81]: np.bincount(train['d']) Out[81]: array([16284, 6207]) In [82]: def cw(df): c0, c1 = np.bincount(df['d']) w0 = (1 / c0) * (len(df)) / 2 w1 = (1 / c1) * (len(df)) / 2 return {0: w0, 1: w1} In [83]: mu, std = train.mean(), train.std() In [84]: train_ = (train - mu) / std In [85]: set_seeds() model = create_model(hl=1, hu=128, reg=True, dropout=False) In [86]: %%time model.fit(train_[cols], train['d'], epochs=40, verbose=False, validation_split=0.2, shuffle=False, class_weight=cw(train)) CPU times: user 40.6 s, sys: 5.49 s, total: 46 s Wall time: 25.2 s

296

|

Chapter 10: Vectorized Backtesting

Out[86]: In [87]: model.evaluate(train_[cols], train['d']) 22491/22491 [==============================] - 0s 13us/step Out[87]: [0.5218664327576152, 0.6729803085327148]

In-sample, the performance looks promising, as illustrated in Figure 10-8: In [88]: train['p'] = np.where(model.predict(train_[cols]) > 0.5, 1, -1) In [89]: train['p'].value_counts() Out[89]: -1 11519 1 10972 Name: p, dtype: int64 In [90]: train['s'] = train['p'] * train['r'] In [91]: train[['r', 's']].sum().apply(np.exp) Out[91]: r 1.0223 s 1.6665 dtype: float64 In [92]: train[['r', 's']].sum().apply(np.exp) - 1 Out[92]: r 0.0223 s 0.6665 dtype: float64 In [93]: train[['r', 's']].cumsum().apply(np.exp).plot(figsize=(10, 6));

Figure 10-8. Gross performance of the passive benchmark investment and the DNN intraday strategy (in-sample) Backtesting an Intraday DNN-Based Strategy

|

297

Out-of-sample, the performance also looks promising before transaction costs. The strategy seems to systematically outperform the passive benchmark investment (see Figure 10-9): In [94]: test = data.iloc[split:].copy() In [95]: test_ = (test - mu) / std In [96]: model.evaluate(test_[cols], test['d']) 3970/3970 [==============================] - 0s 19us/step Out[96]: [0.5226116042706168, 0.668513834476471] In [97]: test['p'] = np.where(model.predict(test_[cols]) > 0.5, 1, -1) In [98]: test['p'].value_counts() Out[98]: -1 2273 1 1697 Name: p, dtype: int64 In [99]: test['s'] = test['p'] * test['r'] In [100]: test[['r', 's']].sum().apply(np.exp) Out[100]: r 1.0071 s 1.0658 dtype: float64 In [101]: test[['r', 's']].sum().apply(np.exp) - 1 Out[101]: r 0.0071 s 0.0658 dtype: float64 In [102]: test[['r', 's']].cumsum().apply(np.exp).plot(figsize=(10, 6));

The final litmus test with regard to pure economic performance comes when adding transaction costs. The strategy leads to hundreds of trades over a relatively short period of time. As the following analysis suggests, based on standard retail bid-ask spreads, the DNN-based strategy is not viable.

298

|

Chapter 10: Vectorized Backtesting

Figure 10-9. Gross performance of the passive benchmark investment and the DNN intraday strategy (out-of-sample) Reducing the spread to a level that professional, high-volume traders might achieve, the strategy still does not break even but rather loses a large proportion of the profits to the transaction costs (see Figure 10-10): In [103]: sum(test['p'].diff() != 0) Out[103]: 1303 In [104]: spread = 0.00012 pc_1 = spread / test[symbol] In [105]: spread = 0.00006 pc_2 = spread / test[symbol] In [106]: test['s_1'] = np.where(test['p'].diff() != 0, test['s'] - pc_1, test['s']) In [107]: test['s_1'].iloc[0] -= pc_1.iloc[0] test['s_1'].iloc[-1] -= pc_1.iloc[0] In [108]: test['s_2'] = np.where(test['p'].diff() != 0, test['s'] - pc_2, test['s']) In [109]: test['s_2'].iloc[0] -= pc_2.iloc[0] test['s_2'].iloc[-1] -= pc_2.iloc[0] In [110]: test[['r', 's', 's_1', 's_2']].sum().apply(np.exp) Out[110]: r 1.0071 s 1.0658

Backtesting an Intraday DNN-Based Strategy

|

299

s_1 0.9259 s_2 0.9934 dtype: float64 In [111]: test[['r', 's', 's_1', 's_2']].sum().apply(np.exp) - 1 Out[111]: r 0.0071 s 0.0658 s_1 -0.0741 s_2 -0.0066 dtype: float64 In [112]: test[['r', 's', 's_1', 's_2']].cumsum().apply( np.exp).plot(figsize=(10, 6), style=['-', '-', '--', '--']);

Assumes bid-ask spread on retail level Assumes bid-ask spread on professional level

Figure 10-10. Gross performance of the DNN intraday strategy before and after higher/ lower transaction costs (out-of-sample)

300

| Chapter 10: Vectorized Backtesting

Intraday Trading Intraday algorithmic trading in the form discussed in this chapter often seems appealing from a statistical point of view. Both in-sample and out-of-sample, the DNN model reaches a high accu‐ racy when predicting the market direction. Excluding transaction costs, this also translates both in-sample and out-of-sample into a significant outperformance of the DNN-based strategy when com‐ pared to the passive benchmark investment. However, adding transaction costs to the mix reduces the performance of the DNNbased strategy considerably, making it unviable for typical retail bid-ask spreads and not really attractive for lower, high-volume bid-ask spreads.

Conclusions Vectorized backtesting proves to be an efficient and valuable approach for backtesting the performance of AI-powered algorithmic trading strategies. This chapter first explains the basic idea behind the approach based on a simple example using two SMAs to derive signals. This allows for a simple visualization of the strategy and resulting positions. It then proceeds by backtesting a DNN-based trading strategy, as discussed in detail in Chapter 7, in combination with EOD data. Both before and after transaction costs, the statistical inefficiencies as discovered in Chapter 7 translate into economic inefficiencies, which means profitable trading strategies. When using the same vectorized backtesting approaches with intraday data, the DNN strategy also shows a significant outperformance both in- and out-of-sample when compared to the passive benchmark investment—at least before transaction costs. Adding trans‐ action costs to the backtesting illustrates that these must be pretty low, on a level often not even achieved by big professional traders, to render the trading strategy economically viable.

References Books and papers cited in this chapter: Gibbs Samuel. 2016. “Elon Musk: Tesla Cars Will Be Able to Cross Us with No Driver in Two Years.” The Guardian. January 11, 2016. https://oreil.ly/C508Q. Hilpisch, Yves. 2018. Python for Finance: Mastering Data-Driven Finance. 2nd ed. Sebastopol: O’Reilly. ⸻. 2020. Python for Algorithmic Trading: From Idea to Cloud Deployment. Sebas‐ topol: O’Reilly.

Conclusions

|

301

CHAPTER 11

Risk Management

A significant barrier to deploying autonomous vehicles (AVs) on a massive scale is safety assurance. —Majid Khonji et al. (2019) Having better prediction raises the value of judgment. After all, it doesn’t help to know the likelihood of rain if you don’t know how much you like staying dry or how much you hate carrying an umbrella. —Ajay Agrawal et al. (2018)

Vectorized backtesting in general enables one to judge the economic potential of a prediction-based algorithmic trading strategy on an as-is basis (that is, in its pure form). Most AI agents applied in practice have more components than just the prediction model. For example, the AI of autonomous vehicles (AVs) comes not standalone but rather with a large number of rules and heuristics that restrict what actions the AI takes or can take. In the context of AVs, this primarily relates to man‐ aging risks, such as those resulting from collisions or crashes. In a financial context, AI agents or trading bots are also not deployed as-is in general. Rather, there are a number of standard risk measures that are typically used, such as (trailing) stop loss orders or take profit orders. The reasoning is clear. When placing directional bets in financial markets, too-large losses are to be avoided. Similarly, when a certain profit level is reached, the success is to be protected by early close outs. How such risk measures are handled is a matter, more often than not, of human judgment, supported probably by a formal analysis of relevant data and statistics. Conceptually, this is a major point discussed in the book by Agrawal et al. (2018): AI provides improved predictions, but human judgment still plays a role in setting deci‐ sion rules and action boundaries.

303

This chapter has a threefold purpose. First, it backtests in both vectorized and eventbased fashion algorithmic trading strategies that result from a trained deep Q-learning agent. Henceforth, such agents are called trading bots. Second, it assesses risks related to the financial instrument on which the strategies are implemented. And third, it backtests typical risk measures, such as stop loss orders, using the eventbased approach introduced in this chapter. The major benefit of event-based backtesting when compared to vectorized backtesting is a higher degree of flexibility in modeling and analyzing decision rules and risk management measures. In other words, it allows one to zoom in on details that are pushed toward the background when working with vectorized programming approaches. “Trading Bot” on page 304 introduces and trains the trading bot based on the finan‐ cial Q-learning agent from Chapter 9. “Vectorized Backtesting” on page 308 uses vec‐ torized backtesting from Chapter 10 to judge the (pure) economic performance of the trading bot. Event-based backtesting is introduced in “Event-Based Backtesting” on page 311. First, a base class is discussed. Second, based on the base class, the backtest‐ ing of the trading bot is implemented and conducted. In this context, also see Hil‐ pisch (2020, ch. 6). “Assessing Risk” on page 318 analyzes selected statistical measures important for setting risk management rules, such as maximum drawdown and aver‐ age true range (ATR). “Backtesting Risk Measures” on page 322 then backtests the impact of major risk measures on the performance of the trading bot.

Trading Bot This section presents a trading bot based on the financial Q-learning agent, FQLAgent, from Chapter 9. This is the trading bot that is analyzed in subsequent sections. As usual, our imports come first: In [1]: import os import numpy as np import pandas as pd from pylab import plt, mpl plt.style.use('seaborn') mpl.rcParams['savefig.dpi'] = 300 mpl.rcParams['font.family'] = 'serif' pd.set_option('mode.chained_assignment', None) pd.set_option('display.float_format', '{:.4f}'.format) np.set_printoptions(suppress=True, precision=4) os.environ['PYTHONHASHSEED'] = '0'

“Finance Environment” on page 333 presents a Python module with the Finance class used in the following. “Trading Bot” on page 304 provides the Python module with the TradingBot class and some helper functions for plotting training and valida‐ tion results. Both classes are pretty close to the ones introduced in Chapter 9, which is why they are used here without further explanations.

304

|

Chapter 11: Risk Management

The following code trains the trading bot on historical end-of-day (EOD) data, including a sub-set of the data used for validation. Figure 11-1 shows average total rewards as achieved for the different training episodes: In [2]: import finance import tradingbot Using TensorFlow backend. In [3]: symbol = 'EUR=' features = [symbol, 'r', 's', 'm', 'v'] In [4]: a = 0 b = 1750 c = 250 In [5]: learn_env = finance.Finance(symbol, features, window=20, lags=3, leverage=1, min_performance=0.9, min_accuracy=0.475, start=a, end=a + b, mu=None, std=None) In [6]: learn_env.data.info()

DatetimeIndex: 1750 entries, 2010-02-02 to 2017-01-12 Data columns (total 6 columns): # Column Non-Null Count Dtype --- ------ -------------- ----0 EUR= 1750 non-null float64 1 r 1750 non-null float64 2 s 1750 non-null float64 3 m 1750 non-null float64 4 v 1750 non-null float64 5 d 1750 non-null int64 dtypes: float64(5), int64(1) memory usage: 95.7 KB In [7]: valid_env = finance.Finance(symbol, features=learn_env.features, window=learn_env.window, lags=learn_env.lags, leverage=learn_env.leverage, min_performance=0.0, min_accuracy=0.0, start=a + b, end=a + b + c, mu=learn_env.mu, std=learn_env.std) In [8]: valid_env.data.info()

DatetimeIndex: 250 entries, 2017-01-13 to 2018-01-10 Data columns (total 6 columns): # Column Non-Null Count Dtype --- ------ -------------- ----0 EUR= 250 non-null float64 1 r 250 non-null float64 2 s 250 non-null float64 3 m 250 non-null float64

Trading Bot

|

305

4 v 250 non-null float64 5 d 250 non-null int64 dtypes: float64(5), int64(1) memory usage: 13.7 KB In [9]: tradingbot.set_seeds(100) agent = tradingbot.TradingBot(24, 0.001, learn_env, valid_env) In [10]: episodes = 61 In [11]: %time agent.learn(episodes) ======================================================================= episode: 10/61 | VALIDATION | treward: 247 | perf: 0.936 | eps: 0.95 ======================================================================= ======================================================================= episode: 20/61 | VALIDATION | treward: 247 | perf: 0.897 | eps: 0.86 ======================================================================= ======================================================================= episode: 30/61 | VALIDATION | treward: 247 | perf: 1.035 | eps: 0.78 ======================================================================= ======================================================================= episode: 40/61 | VALIDATION | treward: 247 | perf: 0.935 | eps: 0.70 ======================================================================= ======================================================================= episode: 50/61 | VALIDATION | treward: 247 | perf: 0.890 | eps: 0.64 ======================================================================= ======================================================================= episode: 60/61 | VALIDATION | treward: 247 | perf: 0.998 | eps: 0.58 ======================================================================= episode: 61/61 | treward: 17 | perf: 0.979 | av: 475.1 | max: 1747 CPU times: user 51.4 s, sys: 2.53 s, total: 53.9 s Wall time: 47 s In [12]: tradingbot.plot_treward(agent)

306

|

Chapter 11: Risk Management

Figure 11-1. Average total reward per training episode Figure 11-2 compares the gross performance of the trading bot on the training data— exhibiting quite some variance due to alternating between exploitation and explora‐ tion—with the one on the validation data set making use of exploitation only: In [13]: tradingbot.plot_performance(agent)

Figure 11-2. Gross performance on training and validation data set This trained trading bot is used for backtesting in the following sections.

Trading Bot

|

307

Vectorized Backtesting Vectorized backtesting cannot directly be applied to the trading bot. Chapter 10 uses dense neural networks (DNNs) to illustrate the approach. In this context, the data with the features and labels sub-sets is prepared first and then fed to the DNN to generate all predictions at once. In a reinforcement learning (RL) context, data is gen‐ erated and collected by interacting with the environment action by action and step by step. To this end, the following Python code defines the backtest function, which takes as input a TradingBot instance and a Finance instance. It generates in the original Data Frame objects of the Finance environment columns with the positions the trading bot takes and the resulting strategy performance: In [14]: def reshape(s): return np.reshape(s, [1, learn_env.lags, learn_env.n_features]) In [15]: def backtest(agent, env): env.min_accuracy = 0.0 env.min_performance = 0.0 done = False env.data['p'] = 0 state = env.reset() while not done: action = np.argmax( agent.model.predict(reshape(state))[0, 0]) position = 1 if action == 1 else -1 env.data.loc[:, 'p'].iloc[env.bar] = position state, reward, done, info = env.step(action) env.data['s'] = env.data['p'] * env.data['r'] * learn_env.leverage

Reshapes a single feature-label combination Generates a column for the position values Derives the optimal action (prediction) given the trained DNN Derives the resulting position (+1 for long/upwards, –1 for short/downwards)… …and stores in the corresponding column at the appropriate index position Calculates the strategy log returns given the position values Equipped with the backtest function, vectorized backtesting boils down to a few lines of Python code as in Chapter 10.

308

|

Chapter 11: Risk Management

Figure 11-3 compares the passive benchmark investment’s gross performance with the strategy gross performance: In [16]: env = agent.learn_env In [17]: backtest(agent, env) In [18]: env.data['p'].iloc[env.lags:].value_counts() Out[18]: 1 961 -1 786 Name: p, dtype: int64 In [19]: env.data[['r', 's']].iloc[env.lags:].sum().apply(np.exp) Out[19]: r 0.7725 s 1.5155 dtype: float64 In [20]: env.data[['r', 's']].iloc[env.lags:].sum().apply(np.exp) - 1 Out[20]: r -0.2275 s 0.5155 dtype: float64 In [21]: env.data[['r', 's']].iloc[env.lags:].cumsum( ).apply(np.exp).plot(figsize=(10, 6));

Specifies the relevant environment Generates the additional data required Counts the number of long and short positions Calculates the gross performances for the passive benchmark investment (r) and the strategy (s)… …as well as the corresponding net performances

Vectorized Backtesting

|

309

Figure 11-3. Gross performance of the passive benchmark investment and the trading bot (in-sample) To get a more realistic picture of the performance of the trading bot, the following Python code creates a test environment with data that the trading bot has not yet seen. Figure 11-4 shows how the trading bot fares compared to the passive benchmark investment: In [22]: test_env = finance.Finance(symbol, features=learn_env.features, window=learn_env.window, lags=learn_env.lags, leverage=learn_env.leverage, min_performance=0.0, min_accuracy=0.0, start=a + b + c, end=None, mu=learn_env.mu, std=learn_env.std) In [23]: env = test_env In [24]: backtest(agent, env) In [25]: env.data['p'].iloc[env.lags:].value_counts() Out[25]: -1 437 1 56 Name: p, dtype: int64 In [26]: env.data[['r', 's']].iloc[env.lags:].sum().apply(np.exp) Out[26]: r 0.9144 s 1.0992 dtype: float64 In [27]: env.data[['r', 's']].iloc[env.lags:].sum().apply(np.exp) - 1 Out[27]: r -0.0856 s 0.0992

310

|

Chapter 11: Risk Management

dtype: float64 In [28]: env.data[['r', 's']].iloc[env.lags:].cumsum( ).apply(np.exp).plot(figsize=(10, 6));

Figure 11-4. Gross performance of the passive benchmark investment and the trading bot (out-of-sample) The out-of-sample performance without any risk measures implemented seems already promising. However, to be able to properly judge the real performance of a trading strategy, risk measures should be included. This is where event-based back‐ testing comes into play.

Event-Based Backtesting Given the results of the previous section, the out-of-sample performance without any risk measures seems already promising. However, to be able to properly analyze risk measures, such as trailing stop loss orders, event-based backtesting is required. This section introduces this alternative approach to judging the performance of algorith‐ mic trading strategies. “Backtesting Base Class” on page 339 presents the BacktestingBase class that can be flexibly used to test different types of directional trading strategies. The code has detailed comments on the important lines. This base class provides the following methods: get_date_price() For a given bar (index value for the DataFrame object containing the financial data), it returns the relevant date and price.

Event-Based Backtesting

|

311

print_balance() For a given bar, it prints the current (cash) balance of the trading bot. calculate_net_wealth() For a given price, it returns the net wealth composed of the current (cash) bal‐

ance and the instrument position.

print_net_wealth() For a given bar, it prints the net wealth of the trading bot. place_buy_order(), place_sell_order() For a given bar and a given number of units or a given amount, these methods

place buy or sell orders and adjust relevant quantities accordingly (for example, accounting for transaction costs).

close_out()

At a given bar, this method closes open positions and calculates and reports per‐ formance statistics. The following Python code illustrates how an instance of the BacktestingBase class functions based on some simple steps: In [29]: import backtesting as bt In [30]: bb = bt.BacktestingBase(env=agent.learn_env, model=agent.model, amount=10000, ptc=0.0001, ftc=1.0, verbose=True) In [31]: bb.initial_amount Out[31]: 10000 In [32]: bar = 100 In [33]: bb.get_date_price(bar) Out[33]: ('2010-06-25', 1.2374) In [34]: bb.env.get_state(bar) Out[34]: EUR= r s m v Date 2010-06-22 -0.0242 -0.5622 -0.0916 -0.2022 1.5316 2010-06-23 0.0176 0.6940 -0.0939 -0.0915 1.5563 2010-06-24 0.0354 0.3034 -0.0865 0.6391 1.0890 In [35]: bb.place_buy_order(bar, amount=5000) 2010-06-25 | buy 4040 units for 1.2374 2010-06-25 | current balance = 4999.40 In [36]: bb.print_net_wealth(2 * bar) 2010-11-16 | net wealth = 10450.17

312

|

Chapter 11: Risk Management

In [37]: bb.place_sell_order(2 * bar, units=1000) 2010-11-16 | sell 1000 units for 1.3492 2010-11-16 | current balance = 6347.47 In [38]: bb.close_out(3 * bar) ================================================== 2011-04-11 | *** CLOSING OUT *** 2011-04-11 | sell 3040 units for 1.4434 2011-04-11 | current balance = 10733.97 2011-04-11 | net performance [%] = 7.3397 2011-04-11 | number of trades [#] = 3 ==================================================

Instantiates a BacktestingBase object Looks up the initial_amount attribute value Fixes a bar value Retrieves the date and price values for the bar Retrieves the state of the Finance environment for the bar Places a buy order using the amount parameter Prints the net wealth at a later point (2 * bar) Places a sell order at that later point using the units parameter Closes out the remaining long position even later (3 * bar) Inheriting from the BacktestingBase class, the TBBacktester class implements the event-based backtesting for the trading bot: In [39]: class TBBacktester(bt.BacktestingBase): def _reshape(self, state): ''' Helper method to reshape state objects. ''' return np.reshape(state, [1, self.env.lags, self.env.n_features]) def backtest_strategy(self): ''' Event-based backtesting of the trading bot's performance. ''' self.units = 0 self.position = 0 self.trades = 0 self.current_balance = self.initial_amount self.net_wealths = list() for bar in range(self.env.lags, len(self.env.data)): date, price = self.get_date_price(bar)

Event-Based Backtesting

|

313

if self.trades == 0: print(50 * '=') print(f'{date} | *** START BACKTEST ***') self.print_balance(bar) print(50 * '=') state = self.env.get_state(bar) action = np.argmax(self.model.predict( self._reshape(state.values))[0, 0]) position = 1 if action == 1 else -1 if self.position in [0, -1] and position == 1: if self.verbose: print(50 * '-') print(f'{date} | *** GOING LONG ***') if self.position == -1: self.place_buy_order(bar - 1, units=-self.units) self.place_buy_order(bar - 1, amount=self.current_balance) if self.verbose: self.print_net_wealth(bar) self.position = 1 elif self.position in [0, 1] and position == -1: if self.verbose: print(50 * '-') print(f'{date} | *** GOING SHORT ***') if self.position == 1: self.place_sell_order(bar - 1, units=self.units) self.place_sell_order(bar - 1, amount=self.current_balance) if self.verbose: self.print_net_wealth(bar) self.position = -1 self.net_wealths.append((date, self.calculate_net_wealth(price))) self.net_wealths = pd.DataFrame(self.net_wealths, columns=['date', 'net_wealth']) self.net_wealths.set_index('date', inplace=True) self.net_wealths.index = pd.DatetimeIndex( self.net_wealths.index) self.close_out(bar)

Retrieves the state of the Finance environment Generates the optimal action (prediction) given the state and the model object Derives the optimal position (long/short) given the optimal action (prediction) Enters a long position if the conditions are met Enters a short position if the conditions are met

314

|

Chapter 11: Risk Management

Collects the net wealth values over time and transforms them into a DataFrame object The application of the TBBacktester class is straightforward, given that the Finance and TradingBot instances are already available. The following code backtests the trading bot first on the learning environment data—without and with transaction costs. Figure 11-5 compares the two cases visually over time: In [40]: env = learn_env In [41]: tb = TBBacktester(env, agent.model, 10000, 0.0, 0, verbose=False) In [42]: tb.backtest_strategy() ================================================== 2010-02-05 | *** START BACKTEST *** 2010-02-05 | current balance = 10000.00 ================================================== ================================================== 2017-01-12 | *** CLOSING OUT *** 2017-01-12 | current balance = 14601.85 2017-01-12 | net performance [%] = 46.0185 2017-01-12 | number of trades [#] = 828 ================================================== In [43]: tb_ = TBBacktester(env, agent.model, 10000, 0.00012, 0.0, verbose=False) In [44]: tb_.backtest_strategy() ================================================== 2010-02-05 | *** START BACKTEST *** 2010-02-05 | current balance = 10000.00 ================================================== ================================================== 2017-01-12 | *** CLOSING OUT *** 2017-01-12 | current balance = 13222.08 2017-01-12 | net performance [%] = 32.2208 2017-01-12 | number of trades [#] = 828 ================================================== In [45]: ax = tb.net_wealths.plot(figsize=(10, 6)) tb_.net_wealths.columns = ['net_wealth (after tc)'] tb_.net_wealths.plot(ax=ax);

Event-based backtest in-sample without transaction costs Event-based backtest in-sample with transaction costs

Event-Based Backtesting

|

315

Figure 11-5. Gross performance of the trading bot before and after transaction costs (in-sample) Figure 11-6 compares the gross performances of the trading bot for the test environ‐ ment data over time—again, before and after transaction costs: In [46]: env = test_env In [47]: tb = TBBacktester(env, agent.model, 10000, 0.0, 0, verbose=False) In [48]: tb.backtest_strategy() ================================================== 2018-01-17 | *** START BACKTEST *** 2018-01-17 | current balance = 10000.00 ================================================== ================================================== 2019-12-31 | *** CLOSING OUT *** 2019-12-31 | current balance = 10936.79 2019-12-31 | net performance [%] = 9.3679 2019-12-31 | number of trades [#] = 186 ================================================== In [49]: tb_ = TBBacktester(env, agent.model, 10000, 0.00012, 0.0, verbose=False) In [50]: tb_.backtest_strategy() ================================================== 2018-01-17 | *** START BACKTEST *** 2018-01-17 | current balance = 10000.00 ================================================== ================================================== 2019-12-31 | *** CLOSING OUT *** 2019-12-31 | current balance = 10695.72

316

| Chapter 11: Risk Management

2019-12-31 | net performance [%] = 6.9572 2019-12-31 | number of trades [#] = 186 ================================================== In [51]: ax = tb.net_wealths.plot(figsize=(10, 6)) tb_.net_wealths.columns = ['net_wealth (after tc)'] tb_.net_wealths.plot(ax=ax);

Event-based backtest out-of-sample without transaction costs Event-based backtest out-of-sample with transaction costs

Figure 11-6. Gross performance of the trading bot before and after transaction costs (out-of-sample) How does the performance before transaction costs from the event-based backtesting compare to the performance from the vectorized backtesting? Figure 11-7 shows the normalized net wealth compared to the gross performance over time. Due to the dif‐ ferent technical approaches, the two time series are not exactly the same but are pretty similar. The performance difference can be mainly explained by the fact that the event-based backtesting assumes the same amount for every position taken. Vec‐ torized backtesting takes compound effects into account, leading to a slightly higher reported performance: In [52]: ax = (tb.net_wealths / tb.net_wealths.iloc[0]).plot(figsize=(10, 6)) tp = env.data[['r', 's']].iloc[env.lags:].cumsum().apply(np.exp) (tp / tp.iloc[0]).plot(ax=ax);

Event-Based Backtesting

|

317

Figure 11-7. Gross performance of the passive benchmark investment and the trading bot (vectorized and event-based backtesting)

Performance Differences The performance numbers from the vectorized and the eventbased backtesting are close but not exactly the same. In the first case, it is assumed that financial instruments are perfectly divisible. Compounding is also done continuously. In the latter case, only full units of the financial instrument are accepted for trading, which is closer to reality. The net wealth calculations are based on price dif‐ ferences. The event-based code as it is used does not, for example, check whether the current balance is large enough to cover a cer‐ tain trade by cash. This is for sure a simplifying assumption, and buying on margin, for instance, may not always be possible. Code adjustments in this regard are easily added to the BacktestingBase class.

Assessing Risk The implementation of risk measures requires the understanding of the risks involved in trading the chosen financial instrument. Therefore, to properly set parameters for risk measures, such as stop loss orders, an assessment of the risk of the underlying instrument is important. There are many approaches available to measure the risk of a financial instrument. There are, for example, nondirected risk measures, such as volatility or average true range (ATR). There are also directed measures, such as maximum drawdown or value-at-risk (VaR).

318

|

Chapter 11: Risk Management

A common practice when setting target levels for stop loss (SL), trailing stop loss (TSL), or take profit orders (TP) is to relate such levels to ATR values.1 The following Python code calculates the ATR in absolute and relative terms for the financial instru‐ ment on which the trading bot is trained and backtested (that is, the EUR/USD exchange rate). The calculations rely on the data from the learning environment and use a typical window length of 14 days (bars). Figure 11-8 shows the calculated values, which vary significantly over time: In [53]: data = pd.DataFrame(learn_env.data[symbol]) In [54]: data.head() Out[54]: EUR= Date 2010-02-02 1.3961 2010-02-03 1.3898 2010-02-04 1.3734 2010-02-05 1.3662 2010-02-08 1.3652 In [55]: window = 14 In [56]: data['min'] = data[symbol].rolling(window).min() In [57]: data['max'] = data[symbol].rolling(window).max() In [58]: data['mami'] = data['max'] - data['min'] In [59]: data['mac'] = abs(data['max'] - data[symbol].shift(1)) In [60]: data['mic'] = abs(data['min'] - data[symbol].shift(1)) In [61]: data['atr'] = np.maximum(data['mami'], data['mac']) In [62]: data['atr'] = np.maximum(data['atr'], data['mic']) In [63]: data['atr%'] = data['atr'] / data[symbol] In [64]: data[['atr', 'atr%']].plot(subplots=True, figsize=(10, 6));

The instrument price column from the original DataFrame object The window length to be used for the calculations The rolling minimum The rolling maximum

1 For more details on the ATR measure, see ATR (1) Investopedia or ATR (2) Investopedia.

Assessing Risk

|

319

The difference between rolling maximum and minimum The absolute difference between rolling maximum and previous day’s price The absolute difference between rolling minimum and previous day’s price The maximum of the max-min difference and the max-price difference The maximum between the previous maximum and the min-price difference (= ATR) The ATR value in percent from the absolute ATR value and the price

Figure 11-8. Average true range (ATR) in absolute (price) and relative (%) terms The code that follows displays the final values for ATR in absolute and relative terms. A typical rule would be to set, for example, the SL level at the entry price minus x times ATR. Depending on the risk appetite of the trader or investor, x might be smaller than 1 or larger. This is where human judgment or formal risk policies come into play. If x = 1, then the SL level is set at about 2% below the entry level: In [65]: data[['atr', 'atr%']].tail() Out[65]: atr atr% Date 2017-01-06 0.0218 0.0207 2017-01-09 0.0218 0.0206 2017-01-10 0.0218 0.0207 2017-01-11 0.0199 0.0188 2017-01-12 0.0206 0.0194

320

|

Chapter 11: Risk Management

However, leverage plays an important role in this context. If a leverage of, say, 10 is used, which is actually quite low for foreign exchange trading, then the ATR numbers need to be multiplied by the leverage. As a consequence, for an assumed ATR factor of 1, the same SL level from before now is to be set at about 20% instead of just 2%. Or, when taking the median value of the ATR from the whole data set, it is set to be at about 25%: In [66]: leverage = 10 In [67]: data[['atr', 'atr%']].tail() * leverage Out[67]: atr atr% Date 2017-01-06 0.2180 0.2070 2017-01-09 0.2180 0.2062 2017-01-10 0.2180 0.2066 2017-01-11 0.1990 0.1881 2017-01-12 0.2060 0.1942 In [68]: data[['atr', 'atr%']].median() * leverage Out[68]: atr 0.3180 atr% 0.2481 dtype: float64

The basic idea behind relating SL or TP levels to ATR is that one should avoid setting them either too low or too high. Consider a 10 times leveraged position for which the ATR is 20%. Setting an SL level of only 3% or 5% might reduce the financial risk for the position, but it introduces the risk of a stop out that happens too early and that is due to typical movements in the financial instrument. Such “typical movements” within certain ranges are often called noise. The SL order should protect, in general, from unfavorable market movements that are larger than typical price movements (noise). The same holds true for a take profit level. If it is set too high, say at three times the ATR level, decent profits might not be secured and positions might remain open for too long until they give up previous profits. Even if formal analyses and mathematical formulas can be used in this context, the setting of such target levels involves, as they say, more art than science. In a financial context, there is quite a degree of freedom for setting such target levels, and human judgment can come to the rescue. In other contexts, such as for AVs, this is different, as no human judgment is needed to instruct the AI to avoid any collisions with human beings.

Assessing Risk

|

321

NonNormality and NonLinearity A margin stop out closes a trading position in cases when the mar‐ gin, or the invested equity, is used up. Assume a leveraged trading position with a margin stop out in place. For a leverage of 10, for example, the margin is 10% equity. An unfavorable move of 10% or larger in the traded instrument eats up all the equity and triggers the close out of the position—a loss of 100% of the equity. A favora‐ ble move of the underlying of, say, 25% leads to a return on equity of 150%. Even if returns of the traded instrument are normally distributed, leverage and margin stop outs lead to nonnormally distributed returns and asymmetric, nonlinear relationships between the traded instrument and the trading position.

Backtesting Risk Measures Having an idea of the ATR of a financial instrument is often a good start for the implementation of risk measures. To be able to properly backtest the effect of the typ‐ ical risk management orders, some adjustments to the BacktestingBase class are helpful. The following Python code presents a new base class—BacktestBaseRM, which inherits from BacktestingBase—that helps in tracking the entry price of the previous trade as well as the maximum and minimum prices since that trade. These values are used to calculate the relevant performance measures during the eventbased backtesting to which SL, TSL, and TP orders relate: # # Event-Based Backtesting # --Base Class (2) # # (c) Dr. Yves J. Hilpisch # from backtesting import *

class BacktestingBaseRM(BacktestingBase): def set_prices(self, price): ''' Sets prices for tracking of performance. To test for e.g. trailing stop loss hit. ''' self.entry_price = price self.min_price = price self.max_price = price def place_buy_order(self, bar, amount=None, units=None, gprice=None): ''' Places a buy order for a given bar and for a given amount or number of units. ''' date, price = self.get_date_price(bar)

322

| Chapter 11: Risk Management

if gprice is not None: price = gprice if units is None: units = int(amount / price) self.current_balance -= (1 + self.ptc) * units * price + self.ftc self.units += units self.trades += 1 self.set_prices(price) if self.verbose: print(f'{date} | buy {units} units for {price:.4f}') self.print_balance(bar) def place_sell_order(self, bar, amount=None, units=None, gprice=None): ''' Places a sell order for a given bar and for a given amount or number of units. ''' date, price = self.get_date_price(bar) if gprice is not None: price = gprice if units is None: units = int(amount / price) self.current_balance += (1 - self.ptc) * units * price - self.ftc self.units -= units self.trades += 1 self.set_prices(price) if self.verbose: print(f'{date} | sell {units} units for {price:.4f}') self.print_balance(bar)

Sets the entry price for the most recent trade Sets the initial minimum price since the most ecent trade Sets the initial maximum price since the most recent trade Sets the relevant prices after a trade is executed Based on this new base class, “Backtesting Class” on page 342 presents a new back‐ testing class, TBBacktesterRM, that allows the inclusion of SL, TSL, and TP orders. The relevant code parts are discussed in the following sub-sections. The parametriza‐ tion of the backtesting examples orients itself roughly on an ATR level of about 2%, as calculated in the previous section.

Backtesting Risk Measures

|

323

EUT and Risk Measures EUT, MVP, and the CAPM (see Chapters 3 and 4) assume that financial agents know about the future distribution of the returns of a financial instrument. MPT and the CAPM assume further‐ more that returns are normally distributed and that there is, for example, a linear relationship between the market portfolio’s returns and the returns of a traded financial instrument. The use of SL, TSL, and TP orders leads—similar and in addition to leverage in combination with margin stop out—to a “guaranteed nonnor‐ mal” distribution and to highly asymmetric, nonlinear payoffs of a trading position in relation to the traded instrument.

Stop Loss The first risk measure is the SL order. It fixes a certain price level or, more often, a fixed percent value that triggers the closing of a position. For example, if the entry price for an unleveraged position is 100 and the SL level is set to 5%, then a long posi‐ tion is closed out at 95 while a short position is closed out at 105. The following Python code is the relevant part of the TBBacktesterRM class that han‐ dles an SL order. For the SL order, the class allows one to specify whether the price level for the order is guaranteed or not.2 Working with guaranteed SL price levels might lead to too-optimistic performance results: # stop loss order if sl is not None and self.position != 0: rc = (price - self.entry_price) / self.entry_price if self.position == 1 and rc < -self.sl: print(50 * '-') if guarantee: price = self.entry_price * (1 - self.sl) print(f'*** STOP LOSS (LONG | {-self.sl:.4f}) ***') else: print(f'*** STOP LOSS (LONG | {rc:.4f}) ***') self.place_sell_order(bar, units=self.units, gprice=price) self.wait = wait self.position = 0 elif self.position == -1 and rc > self.sl: print(50 * '-') if guarantee: price = self.entry_price * (1 + self.sl) print(f'*** STOP LOSS (SHORT | -{self.sl:.4f}) ***') else: print(f'*** STOP LOSS (SHORT | -{rc:.4f}) ***')

2 A guaranteed stop loss order might only be available in certain jurisdictions for certain groups of broker cli‐

ents, such as retail investors/traders.

324

|

Chapter 11: Risk Management

self.place_buy_order(bar, units=-self.units, gprice=price) self.wait = wait self.position = 0

Checks whether an SL is defined and whether the position is not neutral Calculates the performance based on the entry price for the last trade Checks whether an SL event is given for a long position Closes the long position, at either the current price or the guaranteed price level Sets the number of bars to wait before the next trade happens to wait Sets the position to neutral Checks whether an SL event is given for a short position Closes the short position, at either the current price or the guaranteed price level The following Python code backtests the trading strategy of the trading bot without and with an SL order. For the given parametrization, the SL order has a negative impact on the strategy performance: In [69]: import tbbacktesterrm as tbbrm In [70]: env = test_env In [71]: tb = tbbrm.TBBacktesterRM(env, agent.model, 10000, 0.0, 0, verbose=False) In [72]: tb.backtest_strategy(sl=None, tsl=None, tp=None, wait=5) ================================================== 2018-01-17 | *** START BACKTEST *** 2018-01-17 | current balance = 10000.00 ================================================== ================================================== 2019-12-31 | *** CLOSING OUT *** 2019-12-31 | current balance = 10936.79 2019-12-31 | net performance [%] = 9.3679 2019-12-31 | number of trades [#] = 186 ================================================== In [73]: tb.backtest_strategy(sl=0.0175, tsl=None, tp=None, wait=5, guarantee=False) ================================================== 2018-01-17 | *** START BACKTEST *** 2018-01-17 | current balance = 10000.00 ================================================== --------------------------------------------------

Backtesting Risk Measures

|

325

*** STOP LOSS (SHORT | -0.0203) *** ================================================== 2019-12-31 | *** CLOSING OUT *** 2019-12-31 | current balance = 10717.32 2019-12-31 | net performance [%] = 7.1732 2019-12-31 | number of trades [#] = 188 ================================================== In [74]: tb.backtest_strategy(sl=0.017, tsl=None, tp=None, wait=5, guarantee=True) ================================================== 2018-01-17 | *** START BACKTEST *** 2018-01-17 | current balance = 10000.00 ================================================== -------------------------------------------------*** STOP LOSS (SHORT | -0.0170) *** ================================================== 2019-12-31 | *** CLOSING OUT *** 2019-12-31 | current balance = 10753.52 2019-12-31 | net performance [%] = 7.5352 2019-12-31 | number of trades [#] = 188 ==================================================

Instantiates the backtesting class for risk management Backtests the trading bot performance without any risk measure Backtests the trading bot performance with an SL order (no guarantee) Backtests the trading bot performance with an SL order (with guarantee)

Trailing Stop Loss In contrast to a regular SL order, a TSL order is adjusted whenever a new high is observed after the base order has been placed. Assume the base order for an unlever‐ aged long position has an entry price of 95 and the TSL is set to 5%. If the instrument price reaches 100 and falls back to 95, this implies a TSL event, and the position is closed at the entry price level. If the price reaches 110 and falls back to 104.5, this would imply another TSL event. The following Python code is the relevant part of the TBBacktesterRM class that han‐ dles a TSL order. To handle such an order correctly, the maximum prices (highs) and the minimum prices (lows) need to be tracked. The maximum price is relevant for a long position, whereas the minimum price is relevant for a short position: # trailing stop loss order if tsl is not None and self.position != 0: self.max_price = max(self.max_price, price) self.min_price = min(self.min_price, price)

326

|

Chapter 11: Risk Management

rc_1 = (price - self.max_price) / self.entry_price rc_2 = (self.min_price - price) / self.entry_price if self.position == 1 and rc_1 < -self.tsl: print(50 * '-') print(f'*** TRAILING SL (LONG | {rc_1:.4f}) ***') self.place_sell_order(bar, units=self.units) self.wait = wait self.position = 0 elif self.position == -1 and rc_2 < -self.tsl: print(50 * '-') print(f'*** TRAILING SL (SHORT | {rc_2:.4f}) ***') self.place_buy_order(bar, units=-self.units) self.wait = wait self.position = 0

Updates the maximum price if necessary Updates the minimum price if necessary Calculates the relevant performance for a long position Calculates the relevant performance for a short position Checks whether a TSL event is given for a long position Checks whether a TSL event is given for a short position As the backtesting results that follow show, using a TSL order with the given para‐ metrization reduces the gross performance compared to a strategy without a TSL order in place: In [75]: tb.backtest_strategy(sl=None, tsl=0.015, tp=None, wait=5) ================================================== 2018-01-17 | *** START BACKTEST *** 2018-01-17 | current balance = 10000.00 ================================================== -------------------------------------------------*** TRAILING SL (SHORT | -0.0152) *** -------------------------------------------------*** TRAILING SL (SHORT | -0.0169) *** -------------------------------------------------*** TRAILING SL (SHORT | -0.0164) *** -------------------------------------------------*** TRAILING SL (SHORT | -0.0191) *** -------------------------------------------------*** TRAILING SL (SHORT | -0.0166) *** -------------------------------------------------*** TRAILING SL (SHORT | -0.0194) *** --------------------------------------------------

Backtesting Risk Measures

|

327

*** TRAILING SL (SHORT | -0.0172) *** -------------------------------------------------*** TRAILING SL (SHORT | -0.0181) *** -------------------------------------------------*** TRAILING SL (SHORT | -0.0153) *** -------------------------------------------------*** TRAILING SL (SHORT | -0.0160) *** ================================================== 2019-12-31 | *** CLOSING OUT *** 2019-12-31 | current balance = 10577.93 2019-12-31 | net performance [%] = 5.7793 2019-12-31 | number of trades [#] = 201 ==================================================

Backtests the trading bot performance with a TSL order

Take Profit Finally, there are TP orders. A TP order closes out a position that has reached a cer‐ tain profit level. Say an unleveraged long position is opened at a price of 100 and the TP order is set to a level of 5%. If the price reaches 105, the position is closed. The following code from the TBBacktesterRM class finally shows the part that handles a TP order. The TP implementation is straightforward, given the references of the SL and TSL order codes. For the TP order, there is also the option to backtest with a guaranteed price level as compared to the relevant high/low price levels, which would most probably lead to performance values that are too optimistic:3 # take profit order if tp is not None and self.position != 0: rc = (price - self.entry_price) / self.entry_price if self.position == 1 and rc > self.tp: print(50 * '-') if guarantee: price = self.entry_price * (1 + self.tp) print(f'*** TAKE PROFIT (LONG | {self.tp:.4f}) ***') else: print(f'*** TAKE PROFIT (LONG | {rc:.4f}) ***') self.place_sell_order(bar, units=self.units, gprice=price) self.wait = wait self.position = 0 elif self.position == -1 and rc < -self.tp: print(50 * '-') if guarantee: price = self.entry_price * (1 - self.tp) print(f'*** TAKE PROFIT (SHORT | {self.tp:.4f}) ***')

3 A take profit order has a fixed target price level. Therefore, it is unrealistic to use the high price of a time

interval for a long position or the low price of the interval for a short position to calculate the realized profit.

328

|

Chapter 11: Risk Management

else: print(f'*** TAKE PROFIT (SHORT | {-rc:.4f}) ***') self.place_buy_order(bar, units=-self.units, gprice=price) self.wait = wait self.position = 0

For the given parametrization, adding a TP order—without guarantee—improves the trading bot performance noticeably compared to the passive benchmark investment. This result might be too optimistic given the considerations from before. Therefore, the TP order with guarantee leads to a more realistic performance value in this case: In [76]: tb.backtest_strategy(sl=None, tsl=None, tp=0.015, wait=5, guarantee=False) ================================================== 2018-01-17 | *** START BACKTEST *** 2018-01-17 | current balance = 10000.00 ================================================== -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0155) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0155) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0204) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0240) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0168) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0156) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0183) *** ================================================== 2019-12-31 | *** CLOSING OUT *** 2019-12-31 | current balance = 11210.33 2019-12-31 | net performance [%] = 12.1033 2019-12-31 | number of trades [#] = 198 ================================================== In [77]: tb.backtest_strategy(sl=None, tsl=None, tp=0.015, wait=5, guarantee=True) ================================================== 2018-01-17 | *** START BACKTEST *** 2018-01-17 | current balance = 10000.00 ================================================== -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0150) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0150) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0150) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0150) ***

Backtesting Risk Measures

|

329

-------------------------------------------------*** TAKE PROFIT (SHORT | 0.0150) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0150) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0150) *** ================================================== 2019-12-31 | *** CLOSING OUT *** 2019-12-31 | current balance = 10980.86 2019-12-31 | net performance [%] = 9.8086 2019-12-31 | number of trades [#] = 198 ==================================================

Backtests the trading bot performance with a TP order (no guarantee) Backtests the trading bot performance with a TP order (with guarantee) Of course, SL/TSL orders can also be combined with TP orders. The backtest results of the Python code that follows are in both cases worse than those for the strategy without the risk measures in place. In managing risk, there is hardly any free lunch: In [78]: tb.backtest_strategy(sl=0.015, tsl=None, tp=0.0185, wait=5) ================================================== 2018-01-17 | *** START BACKTEST *** 2018-01-17 | current balance = 10000.00 ================================================== -------------------------------------------------*** STOP LOSS (SHORT | -0.0203) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0202) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0213) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0240) *** -------------------------------------------------*** STOP LOSS (SHORT | -0.0171) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0188) *** -------------------------------------------------*** STOP LOSS (SHORT | -0.0153) *** -------------------------------------------------*** STOP LOSS (SHORT | -0.0154) *** ================================================== 2019-12-31 | *** CLOSING OUT *** 2019-12-31 | current balance = 10552.00 2019-12-31 | net performance [%] = 5.5200 2019-12-31 | number of trades [#] = 201 ================================================== In [79]: tb.backtest_strategy(sl=None, tsl=0.02, tp=0.02, wait=5)

330

|

Chapter 11: Risk Management

================================================== 2018-01-17 | *** START BACKTEST *** 2018-01-17 | current balance = 10000.00 ================================================== -------------------------------------------------*** TRAILING SL (SHORT | -0.0235) *** -------------------------------------------------*** TRAILING SL (SHORT | -0.0202) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0250) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0227) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0240) *** -------------------------------------------------*** TRAILING SL (SHORT | -0.0216) *** -------------------------------------------------*** TAKE PROFIT (SHORT | 0.0241) *** -------------------------------------------------*** TRAILING SL (SHORT | -0.0206) *** ================================================== 2019-12-31 | *** CLOSING OUT *** 2019-12-31 | current balance = 10346.38 2019-12-31 | net performance [%] = 3.4638 2019-12-31 | number of trades [#] = 198 ==================================================

Backtests the trading bot performance with an SL and TP order Backtests the trading bot performance with a TSL and TP order

Performance Impact Risk measures have their reasoning and benefits. However, reduc‐ ing risk may come at the price of lower overall performance. On the other hand, the backtesting example with the TP order shows performance improvements that can be explained by the fact that, given the ATR of a financial instrument, a certain profit level can be considered good enough to realize the profit. Any hope to see even higher profits typically is smashed by the market turning around again.

Backtesting Risk Measures

|

331

Conclusions This chapter has three main topics. It backtests the performance of a trading bot (that is, a trained deep Q-learning agent) out-of-sample in both vectorized and event-based fashion. It also assesses risks in the form of the average true range (ATR) indicator that measures the typical variation in the price of the financial instrument of interest. Finally, the chapter discusses and backtests event-based typical risk measures in the form of stop loss (SL), trailing stop loss (TSL), and take profit (TP) orders. Similar to autonomous vehicles (AVs), trading bots are hardly ever deployed based on the predictions of their AI only. To avoid large downside risks and to improve the (risk-adjusted) performance, risk measures usually come into play. Standard risk measures, as discussed in this chapter, are available on almost every trading platform, as well as for retail traders. The next chapter illustrates this in the context of the Oanda trading platform. The event-based backtesting approach provides the algorith‐ mic flexibility to properly backtest the effects of such risk measures. While “reducing risk” may sound appealing, the backtest results indicate that the reduction in risk often comes at a cost: the performance might be lower when compared to the pure strategy without any risk measures. However, when finely tuned, the results also show that TP orders, for example, can also have a positive effect on the performance.

References Books and papers cited in this chapter: Agrawal, Ajay, Joshua Gans, and Avi Goldfarb. 2018. Prediction Machines: The Simple Economics of Artificial Intelligence. Boston: Harvard Business Review Press. Hilpisch, Yves. 2020. Python for Algorithmic Trading: From Idea to Cloud Deployment. Sebastopol: O’Reilly. Khonji, Majid, Jorge Dias, and Lakmal Seneviratne. 2019. “Risk-Aware Reasoning for Autonomous Vehicles.” arXiv. October 6, 2019. https://oreil.ly/2Z6WR.

332

|

Chapter 11: Risk Management

Python Code Finance Environment The following is the Python module with the Finance environment class: # # Finance Environment # # (c) Dr. Yves J. Hilpisch # Artificial Intelligence in Finance # import math import random import numpy as np import pandas as pd

class observation_space: def __init__(self, n): self.shape = (n,)

class action_space: def __init__(self, n): self.n = n def sample(self): return random.randint(0, self.n - 1)

class Finance: intraday = False if intraday: url = 'http://hilpisch.com/aiif_eikon_id_eur_usd.csv' else: url = 'http://hilpisch.com/aiif_eikon_eod_data.csv' def __init__(self, symbol, features, window, lags, leverage=1, min_performance=0.85, min_accuracy=0.5, start=0, end=None, mu=None, std=None): self.symbol = symbol self.features = features self.n_features = len(features) self.window = window self.lags = lags self.leverage = leverage self.min_performance = min_performance self.min_accuracy = min_accuracy self.start = start self.end = end

Python Code

|

333

self.mu = mu self.std = std self.observation_space = observation_space(self.lags) self.action_space = action_space(2) self._get_data() self._prepare_data() def _get_data(self): self.raw = pd.read_csv(self.url, index_col=0, parse_dates=True).dropna() if self.intraday: self.raw = self.raw.resample('30min', label='right').last() self.raw = pd.DataFrame(self.raw['CLOSE']) self.raw.columns = [self.symbol] def _prepare_data(self): self.data = pd.DataFrame(self.raw[self.symbol]) self.data = self.data.iloc[self.start:] self.data['r'] = np.log(self.data / self.data.shift(1)) self.data.dropna(inplace=True) self.data['s'] = self.data[self.symbol].rolling(self.window).mean() self.data['m'] = self.data['r'].rolling(self.window).mean() self.data['v'] = self.data['r'].rolling(self.window).std() self.data.dropna(inplace=True) if self.mu is None: self.mu = self.data.mean() self.std = self.data.std() self.data_ = (self.data - self.mu) / self.std self.data['d'] = np.where(self.data['r'] > 0, 1, 0) self.data['d'] = self.data['d'].astype(int) if self.end is not None: self.data = self.data.iloc[:self.end - self.start] self.data_ = self.data_.iloc[:self.end - self.start] def _get_state(self): return self.data_[self.features].iloc[self.bar self.lags:self.bar] def get_state(self, bar): return self.data_[self.features].iloc[bar - self.lags:bar] def seed(self, seed): random.seed(seed) np.random.seed(seed) def reset(self): self.treward = 0 self.accuracy = 0 self.performance = 1 self.bar = self.lags state = self.data_[self.features].iloc[self.bar self.lags:self.bar]

334

|

Chapter 11: Risk Management

return state.values def step(self, action): correct = action == self.data['d'].iloc[self.bar] ret = self.data['r'].iloc[self.bar] * self.leverage reward_1 = 1 if correct else 0 reward_2 = abs(ret) if correct else -abs(ret) self.treward += reward_1 self.bar += 1 self.accuracy = self.treward / (self.bar - self.lags) self.performance *= math.exp(reward_2) if self.bar >= len(self.data): done = True elif reward_1 == 1: done = False elif (self.performance < self.min_performance and self.bar > self.lags + 15): done = True elif (self.accuracy < self.min_accuracy and self.bar > self.lags + 15): done = True else: done = False state = self._get_state() info = {} return state.values, reward_1 + reward_2 * 5, done, info

Trading Bot The following is the Python module with the TradingBot class, based on a financial Q-learning agent: # # Financial Q-Learning Agent # # (c) Dr. Yves J. Hilpisch # Artificial Intelligence in Finance # import os import random import numpy as np from pylab import plt, mpl from collections import deque import tensorflow as tf from keras.layers import Dense, Dropout from keras.models import Sequential from keras.optimizers import Adam, RMSprop os.environ['PYTHONHASHSEED'] = '0' plt.style.use('seaborn') mpl.rcParams['savefig.dpi'] = 300 mpl.rcParams['font.family'] = 'serif'

Python Code

|

335

def set_seeds(seed=100): ''' Function to set seeds for all random number generators. ''' random.seed(seed) np.random.seed(seed) tf.random.set_seed(seed)

class TradingBot: def __init__(self, hidden_units, learning_rate, learn_env, valid_env=None, val=True, dropout=False): self.learn_env = learn_env self.valid_env = valid_env self.val = val self.epsilon = 1.0 self.epsilon_min = 0.1 self.epsilon_decay = 0.99 self.learning_rate = learning_rate self.gamma = 0.5 self.batch_size = 128 self.max_treward = 0 self.averages = list() self.trewards = [] self.performances = list() self.aperformances = list() self.vperformances = list() self.memory = deque(maxlen=2000) self.model = self._build_model(hidden_units, learning_rate, dropout) def _build_model(self, hu, lr, dropout): ''' Method to create the DNN model. ''' model = Sequential() model.add(Dense(hu, input_shape=( self.learn_env.lags, self.learn_env.n_features), activation='relu')) if dropout: model.add(Dropout(0.3, seed=100)) model.add(Dense(hu, activation='relu')) if dropout: model.add(Dropout(0.3, seed=100)) model.add(Dense(2, activation='linear')) model.compile( loss='mse', optimizer=RMSprop(lr=lr) ) return model

336

|

Chapter 11: Risk Management

def act(self, state): ''' Method for taking action based on a) exploration b) exploitation ''' if random.random() self.epsilon_min: self.epsilon *= self.epsilon_decay def learn(self, episodes): ''' Method to train the DQL agent. ''' for e in range(1, episodes + 1): state = self.learn_env.reset() state = np.reshape(state, [1, self.learn_env.lags, self.learn_env.n_features]) for _ in range(10000): action = self.act(state) next_state, reward, done, info = self.learn_env.step(action) next_state = np.reshape(next_state, [1, self.learn_env.lags, self.learn_env.n_features]) self.memory.append([state, action, reward, next_state, done]) state = next_state if done: treward = _ + 1 self.trewards.append(treward) av = sum(self.trewards[-25:]) / 25 perf = self.learn_env.performance self.averages.append(av) self.performances.append(perf) self.aperformances.append( sum(self.performances[-25:]) / 25) self.max_treward = max(self.max_treward, treward)

Python Code

|

337

templ = 'episode: {:2d}/{} | treward: {:4d} | ' templ += 'perf: {:5.3f} | av: {:5.1f} | max: {:4d}' print(templ.format(e, episodes, treward, perf, av, self.max_treward), end='\r') break if self.val: self.validate(e, episodes) if len(self.memory) > self.batch_size: self.replay() print() def validate(self, e, episodes): ''' Method to validate the performance of the DQL agent. ''' state = self.valid_env.reset() state = np.reshape(state, [1, self.valid_env.lags, self.valid_env.n_features]) for _ in range(10000): action = np.argmax(self.model.predict(state)[0, 0]) next_state, reward, done, info = self.valid_env.step(action) state = np.reshape(next_state, [1, self.valid_env.lags, self.valid_env.n_features]) if done: treward = _ + 1 perf = self.valid_env.performance self.vperformances.append(perf) if e % int(episodes / 6) == 0: templ = 71 * '=' templ += '\nepisode: {:2d}/{} | VALIDATION | ' templ += 'treward: {:4d} | perf: {:5.3f} | eps: {:.2f}\n' templ += 71 * '=' print(templ.format(e, episodes, treward, perf, self.epsilon)) break

def plot_treward(agent): ''' Function to plot the total reward per training episode. ''' plt.figure(figsize=(10, 6)) x = range(1, len(agent.averages) + 1) y = np.polyval(np.polyfit(x, agent.averages, deg=3), x) plt.plot(x, agent.averages, label='moving average') plt.plot(x, y, 'r--', label='regression') plt.xlabel('episodes') plt.ylabel('total reward') plt.legend()

def plot_performance(agent):

338

|

Chapter 11: Risk Management

''' Function to plot the financial gross performance per training episode. ''' plt.figure(figsize=(10, 6)) x = range(1, len(agent.performances) + 1) y = np.polyval(np.polyfit(x, agent.performances, deg=3), x) plt.plot(x, agent.performances[:], label='training') plt.plot(x, y, 'r--', label='regression (train)') if agent.val: y_ = np.polyval(np.polyfit(x, agent.vperformances, deg=3), x) plt.plot(x, agent.vperformances[:], label='validation') plt.plot(x, y_, 'r-.', label='regression (valid)') plt.xlabel('episodes') plt.ylabel('gross performance') plt.legend()

Backtesting Base Class The following is the Python module with the BacktestingBase class for event-based backtesting: # # # # # # #

Event-Based Backtesting --Base Class (1) (c) Dr. Yves J. Hilpisch Artificial Intelligence in Finance

class BacktestingBase: def __init__(self, env, model, amount, ptc, ftc, verbose=False): self.env = env self.model = model self.initial_amount = amount self.current_balance = amount self.ptc = ptc self.ftc = ftc self.verbose = verbose self.units = 0 self.trades = 0 def get_date_price(self, bar): ''' Returns date and price for a given bar. ''' date = str(self.env.data.index[bar])[:10] price = self.env.data[self.env.symbol].iloc[bar] return date, price def print_balance(self, bar): ''' Prints the current cash balance for a given bar. '''

Python Code

|

339

date, price = self.get_date_price(bar) print(f'{date} | current balance = {self.current_balance:.2f}') def calculate_net_wealth(self, price): return self.current_balance + self.units * price def print_net_wealth(self, bar): ''' Prints the net wealth for a given bar (cash + position). ''' date, price = self.get_date_price(bar) net_wealth = self.calculate_net_wealth(price) print(f'{date} | net wealth = {net_wealth:.2f}') def place_buy_order(self, bar, amount=None, units=None): ''' Places a buy order for a given bar and for a given amount or number of units. ''' date, price = self.get_date_price(bar) if units is None: units = int(amount / price) # units = amount / price self.current_balance -= (1 + self.ptc) * \ units * price + self.ftc self.units += units self.trades += 1 if self.verbose: print(f'{date} | buy {units} units for {price:.4f}') self.print_balance(bar) def place_sell_order(self, bar, amount=None, units=None): ''' Places a sell order for a given bar and for a given amount or number of units. ''' date, price = self.get_date_price(bar) if units is None: units = int(amount / price) # units = amount / price self.current_balance += (1 - self.ptc) * \ units * price - self.ftc self.units -= units self.trades += 1 if self.verbose: print(f'{date} | sell {units} units for {price:.4f}') self.print_balance(bar) def close_out(self, bar): ''' Closes out any open position at a given bar. ''' date, price = self.get_date_price(bar) print(50 * '=') print(f'{date} | *** CLOSING OUT ***')

340

|

Chapter 11: Risk Management

if self.units < 0: self.place_buy_order(bar, units=-self.units) else: self.place_sell_order(bar, units=self.units) if not self.verbose: print(f'{date} | current balance = {self.current_balance:.2f}') perf = (self.current_balance / self.initial_amount - 1) * 100 print(f'{date} | net performance [%] = {perf:.4f}') print(f'{date} | number of trades [#] = {self.trades}') print(50 * '=')

The relevant Finance environment The relevant DNN model (from the trading bot) The initial/current balance Proportional transaction costs Fixed transaction costs Whether the prints are verbose or not The initial number of units of the financial instrument traded The initial number of trades implemented The relevant date given a certain bar The relevant instrument price at a certain bar The output of the date and current balance for a certain bar The calculation of the net wealth from the current balance and the instrument position The output of the date and the net wealth at a certain bar The number of units to be traded given the trade amount The impact of the trade and the associated costs on the current balance The adjustment of the number of units held The adjustment of the number of trades implemented

Python Code

|

341

The closing of a short position… …or of a long position The net performance given the initial amount and the final current balance

Backtesting Class The following is the Python module with the TBBacktesterRM class for event-based backtesting including risk measures (stop loss, trailing stop loss, take profit orders): # # Event-Based Backtesting # --Trading Bot Backtester # (incl. Risk Management) # # (c) Dr. Yves J. Hilpisch # import numpy as np import pandas as pd import backtestingrm as btr

class TBBacktesterRM(btr.BacktestingBaseRM): def _reshape(self, state): ''' Helper method to reshape state objects. ''' return np.reshape(state, [1, self.env.lags, self.env.n_features]) def backtest_strategy(self, sl=None, tsl=None, tp=None, wait=5, guarantee=False): ''' Event-based backtesting of the trading bot's performance. Incl. stop loss, trailing stop loss and take profit. ''' self.units = 0 self.position = 0 self.trades = 0 self.sl = sl self.tsl = tsl self.tp = tp self.wait = 0 self.current_balance = self.initial_amount self.net_wealths = list() for bar in range(self.env.lags, len(self.env.data)): self.wait = max(0, self.wait - 1) date, price = self.get_date_price(bar) if self.trades == 0: print(50 * '=') print(f'{date} | *** START BACKTEST ***') self.print_balance(bar) print(50 * '=')

342

| Chapter 11: Risk Management

# stop loss order if sl is not None and self.position != 0: rc = (price - self.entry_price) / self.entry_price if self.position == 1 and rc < -self.sl: print(50 * '-') if guarantee: price = self.entry_price * (1 - self.sl) print(f'*** STOP LOSS (LONG | {-self.sl:.4f}) ***') else: print(f'*** STOP LOSS (LONG | {rc:.4f}) ***') self.place_sell_order(bar, units=self.units, gprice=price) self.wait = wait self.position = 0 elif self.position == -1 and rc > self.sl: print(50 * '-') if guarantee: price = self.entry_price * (1 + self.sl) print(f'*** STOP LOSS (SHORT | -{self.sl:.4f}) ***') else: print(f'*** STOP LOSS (SHORT | -{rc:.4f}) ***') self.place_buy_order(bar, units=-self.units, gprice=price) self.wait = wait self.position = 0 # trailing stop loss order if tsl is not None and self.position != 0: self.max_price = max(self.max_price, price) self.min_price = min(self.min_price, price) rc_1 = (price - self.max_price) / self.entry_price rc_2 = (self.min_price - price) / self.entry_price if self.position == 1 and rc_1 < -self.tsl: print(50 * '-') print(f'*** TRAILING SL (LONG | {rc_1:.4f}) ***') self.place_sell_order(bar, units=self.units) self.wait = wait self.position = 0 elif self.position == -1 and rc_2 < -self.tsl: print(50 * '-') print(f'*** TRAILING SL (SHORT | {rc_2:.4f}) ***') self.place_buy_order(bar, units=-self.units) self.wait = wait self.position = 0 # take profit order if tp is not None and self.position != 0: rc = (price - self.entry_price) / self.entry_price if self.position == 1 and rc > self.tp: print(50 * '-') if guarantee: price = self.entry_price * (1 + self.tp) print(f'*** TAKE PROFIT (LONG | {self.tp:.4f}) ***')

Python Code

|

343

else: print(f'*** TAKE PROFIT (LONG | {rc:.4f}) ***') self.place_sell_order(bar, units=self.units, gprice=price) self.wait = wait self.position = 0 elif self.position == -1 and rc < -self.tp: print(50 * '-') if guarantee: price = self.entry_price * (1 - self.tp) print(f'*** TAKE PROFIT (SHORT | {self.tp:.4f}) ***') else: print(f'*** TAKE PROFIT (SHORT | {-rc:.4f}) ***') self.place_buy_order(bar, units=-self.units, gprice=price) self.wait = wait self.position = 0 state = self.env.get_state(bar) action = np.argmax(self.model.predict( self._reshape(state.values))[0, 0]) position = 1 if action == 1 else -1 if self.position in [0, -1] and position == 1 and self.wait == 0: if self.verbose: print(50 * '-') print(f'{date} | *** GOING LONG ***') if self.position == -1: self.place_buy_order(bar - 1, units=-self.units) self.place_buy_order(bar - 1, amount=self.current_balance) if self.verbose: self.print_net_wealth(bar) self.position = 1 elif self.position in [0, 1] and position == -1 and self.wait == 0: if self.verbose: print(50 * '-') print(f'{date} | *** GOING SHORT ***') if self.position == 1: self.place_sell_order(bar - 1, units=self.units) self.place_sell_order(bar - 1, amount=self.current_balance) if self.verbose: self.print_net_wealth(bar) self.position = -1 self.net_wealths.append((date, self.calculate_net_wealth(price))) self.net_wealths = pd.DataFrame(self.net_wealths, columns=['date', 'net_wealth']) self.net_wealths.set_index('date', inplace=True) self.net_wealths.index = pd.DatetimeIndex(self.net_wealths.index) self.close_out(bar)

344

| Chapter 11: Risk Management

CHAPTER 12

Execution and Deployment

Considerable progress is needed before autonomous vehicles can operate reliably in mixed urban traffic, heavy rain and snow, unpaved and unmapped roads, and where wireless access is unreliable. —Todd Litman (2020) An investment firm that engages in algorithmic trading shall have in place effective systems and risk controls suitable to the business it operates to ensure that its trading systems are resilient and have sufficient capacity, are subject to appropriate trading thresholds and limits and prevent the sending of erroneous orders or the systems otherwise functioning in a way that may create or contribute to a disorderly market. —MiFID II (Article 17)

Chapter 11 trains a trading bot in the form of a financial Q-learning agent based on historical data. It introduces event-based backtesting as an approach flexible enough to account for typical risk measures, such as trailing stop loss orders or take profit targets. However, all this happens asynchronously in a sandbox environment based on historical data only. As with an autonomous vehicle (AV), there is the problem of deploying the AI in the real world. For an AV this means combining the AI with the car hardware and deploying the AV on test and public streets. For a trading bot this means connecting the trading bot with a trading platform and deploying it such that orders are executed automatically. In other words, the algorithmic side is clear—exe‐ cution and deployment now need to be added to implement algorithmic trading. This chapter introduces the Oanda trading platform for algorithmic trading. There‐ fore, the focus is on the v20 API of the platform and not on applications that provide users with an interface for manual trading. To simplify the code, the wrapper package tpqoa is introduced and used. It relies on the v20 Python package from Oanda and provides a more Pythonic user interface.

345

“Oanda Account” on page 346 details the prerequisites to use a demo account with Oanda. “Data Retrieval” on page 347 shows how to retrieve historical and real-time (streaming) data from the API. “Order Execution” on page 351 deals with the execu‐ tion of buy and sell orders, potentially including other orders, such as trailing stop loss orders. “Trading Bot” on page 357 trains a trading bot based on historical intra‐ day data from Oanda and backtests its performance in vectorized fashion. Finally, “Deployment” on page 364 shows how to deploy the trading bot in real-time and an automated fashion.

Oanda Account The code in this chapter relies on the Python wrapper package tpqoa. This package can be installed via pip as follows: pip install --upgrade git+https://github.com/yhilpisch/tpqoa.git

To make use of this package, a demo account with Oanda is sufficient. Once the account is open, an access token is generated on the account page (after login). The access token and the account id (also found on the account page) are then stored in a configuration text file as follows: [oanda] account_id = XYZ-ABC-... access_token = ZYXCAB... account_type = practice

If the name of the configuration file is aiif.cfg and if it is stored in the current working directory, then the tpqoa package can be used as follows: import tpqoa api = tpqoa.tpqoa('aiif.cfg')

Risk Disclaimers and Disclosures Oanda is a platform for foreign exchange (FX) and contracts for dif‐ ference (CFD) trading. These instruments involve considerable risks, in particular when traded with leverage. It is strongly recom‐ mended that you read all relevant risk disclaimers and disclosures from Oanda on its website carefully before moving on (check for the appropriate jurisdiction). All code and examples presented in this chapter are for technical illustration only and do not constitute any investment advice or similar.

346

|

Chapter 12: Execution and Deployment

Data Retrieval As usual, some Python imports and configurations come first: In [1]: import os import time import numpy as np import pandas as pd from pprint import pprint from pylab import plt, mpl plt.style.use('seaborn') mpl.rcParams['savefig.dpi'] = 300 mpl.rcParams['font.family'] = 'serif' pd.set_option('mode.chained_assignment', None) pd.set_option('display.float_format', '{:.5f}'.format) np.set_printoptions(suppress=True, precision=4) os.environ['PYTHONHASHSEED'] = '0'

Depending on the relevant jurisdiction of the account, Oanda offers a number of tradable FX and CFD instruments. The following Python code retrieves the available instruments for a given account: In [2]: import tpqoa In [3]: api = tpqoa.tpqoa('../aiif.cfg') In [4]: ins = api.get_instruments() In [5]: ins[:5] Out[5]: [('AUD/CAD', ('AUD/CHF', ('AUD/HKD', ('AUD/JPY', ('AUD/NZD',

'AUD_CAD'), 'AUD_CHF'), 'AUD_HKD'), 'AUD_JPY'), 'AUD_NZD')]

Imports the tpqoa package Instantiates an API object given the account credentials Retrieves the list of available instruments in the format (display_name, technical_name) Shows a select few of these instruments Oanda provides a wealth of historical data via its v20 API. The following examples retrieve historical data for the EUR/USD currency pair—the granularity is set to D (that is, daily).

Data Retrieval

|

347

Figure 12-1 plots the closing (ask) prices: In [6]: raw = api.get_history(instrument='EUR_USD', start='2018-01-01', end='2020-07-31', granularity='D', price='A') In [7]: raw.info()

DatetimeIndex: 671 entries, 2018-01-01 22:00:00 to 2020-07-30 21:00:00 Data columns (total 6 columns): # Column Non-Null Count Dtype --- ------------------- ----0 o 671 non-null float64 1 h 671 non-null float64 2 l 671 non-null float64 3 c 671 non-null float64 4 volume 671 non-null int64 5 complete 671 non-null bool dtypes: bool(1), float64(4), int64(1) memory usage: 32.1 KB In [8]: raw.head() Out[8]: time 2018-01-01 2018-01-02 2018-01-03 2018-01-04 2018-01-07

22:00:00 22:00:00 22:00:00 22:00:00 22:00:00

o

h

l

1.20101 1.20620 1.20170 1.20692 1.20301

1.20819 1.20673 1.20897 1.20847 1.20530

1.20051 1.20018 1.20049 1.20215 1.19564

In [9]: raw['c'].plot(figsize=(10, 6));

Specifies the instrument… …the starting date… …the end date… …the granularity (D = daily)… …and the type of the price series (A = ask)

348

|

Chapter 12: Execution and Deployment

c volume 1.20610 1.20170 1.20710 1.20327 1.19717

35630 31354 35187 36478 27618

complete True True True True True

Figure 12-1. Historical daily closing prices for EUR/USD from Oanda Intraday data is as easily retrieved and used as daily data, as the code that follows shows. Figure 12-2 visualizes minute bar (mid) price data: In [10]: raw = api.get_history(instrument='EUR_USD', start='2020-07-01', end='2020-07-31', granularity='M1', price='M') In [11]: raw.info()

DatetimeIndex: 30728 entries, 2020-07-01 00:00:00 to 2020-07-30 23:59:00 Data columns (total 6 columns): # Column Non-Null Count Dtype --- ------------------- ----0 o 30728 non-null float64 1 h 30728 non-null float64 2 l 30728 non-null float64 3 c 30728 non-null float64 4 volume 30728 non-null int64 5 complete 30728 non-null bool dtypes: bool(1), float64(4), int64(1) memory usage: 1.4 MB In [12]: raw.tail() Out[12]: time 2020-07-30 2020-07-30 2020-07-30 2020-07-30 2020-07-30

23:55:00 23:56:00 23:57:00 23:58:00 23:59:00

o

h

l

c

volume

complete

1.18724 1.18736 1.18756 1.18736 1.18718

1.18739 1.18758 1.18756 1.18737 1.18724

1.18718 1.18722 1.18734 1.18713 1.18714

1.18738 1.18757 1.18734 1.18717 1.18722

57 57 49 36 31

True True True True True

Data Retrieval

|

349

In [13]: raw['c'].plot(figsize=(10, 6));

Specifies the granularity (M1 = one minute)… …and the type of the price series (M = mid)

Figure 12-2. Historical one-minute bar closing prices for EUR/USD from Oanda Whereas historical data is important, for instance, to train and test a trading bot, realtime (streaming) data is required to deploy such a bot for algorithmic trading. tpqoa allows the synchronous streaming of real-time data for all available instruments with a single method call. The method prints by default the time stamp and the bid/ask prices. For algorithmic trading, this default behavior can be adjusted, as “Deploy‐ ment” on page 364 shows: In [14]: api.stream_data('EUR_USD', stop=10) 2020-08-13T12:07:09.735715316Z 1.18328 2020-08-13T12:07:16.245253689Z 1.18329 2020-08-13T12:07:16.397803785Z 1.18328 2020-08-13T12:07:17.240232521Z 1.18331 2020-08-13T12:07:17.358476854Z 1.18334 2020-08-13T12:07:17.778061207Z 1.18331 2020-08-13T12:07:18.016544856Z 1.18333 2020-08-13T12:07:18.144762415Z 1.18334 2020-08-13T12:07:18.689365678Z 1.18331 2020-08-13T12:07:19.148039139Z 1.18331

350

|

Chapter 12: Execution and Deployment

1.18342 1.18343 1.18342 1.18346 1.18348 1.18345 1.18346 1.18348 1.18345 1.18345

Order Execution The AI of an AV needs to be able to control the physical vehicle. To this end it sends different types of signals to the vehicle, for example, to accelerate, break, turn left, or turn right. A trading bot needs to be able to place orders with the trading platform. This section covers different types of orders, such as market orders and stop loss orders. The most fundamental type of order is a market order. This order allows buying or selling a financial instrument at the current market price (that is, the ask price when buying and the bid price when selling). The following examples are based on an account leverage of 20 and relatively small order sizes. Therefore, liquidity issues, for example, do not play a role. When executing orders via the Oanda v20 API, the API returns a detailed order object. First, a buy market order is placed: In [15]: order = api.create_order('EUR_USD', units=25000, suppress=True, ret=True) pprint(order) {'accountBalance': '98553.3172', 'accountID': '101-004-13834683-001', 'batchID': '1625', 'commission': '0.0', 'financing': '0.0', 'fullPrice': {'asks': [{'liquidity': '10000000', 'price': 1.18345}], 'bids': [{'liquidity': '10000000', 'price': 1.18331}], 'closeoutAsk': 1.18345, 'closeoutBid': 1.18331, 'type': 'PRICE'}, 'fullVWAP': 1.18345, 'gainQuoteHomeConversionFactor': '0.840811914585', 'guaranteedExecutionFee': '0.0', 'halfSpreadCost': '1.4788', 'id': '1626', 'instrument': 'EUR_USD', 'lossQuoteHomeConversionFactor': '0.849262285586', 'orderID': '1625', 'pl': '0.0', 'price': 1.18345, 'reason': 'MARKET_ORDER', 'requestID': '78757241547812154', 'time': '2020-08-13T12:07:19.434407966Z', 'tradeOpened': {'guaranteedExecutionFee': '0.0', 'halfSpreadCost': '1.4788', 'initialMarginRequired': '832.5', 'price': 1.18345, 'tradeID': '1626', 'units': '25000.0'}, 'type': 'ORDER_FILL', 'units': '25000.0', 'userID': 13834683}

Order Execution

|

351

In [16]: def print_details(order): details = (order['time'][:-7], order['instrument'], order['units'], order['price'], order['pl']) return details In [17]: print_details(order) Out[17]: ('2020-08-13T12:07:19.434', 'EUR_USD', '25000.0', 1.18345, '0.0') In [18]: time.sleep(1)

Places a buy market order and prints the order object details Selects and shows the time, instrument, units, price, and pl details of the order Second, the position is closed via a sell market order of the same size. Whereas the first trade has a profit/loss (P&L) of zero by its nature—before accounting for trans‐ action costs—the second trade in general has a nonzero P&L: In [19]: order = api.create_order('EUR_USD', units=-25000, suppress=True, ret=True) pprint(order) {'accountBalance': '98549.283', 'accountID': '101-004-13834683-001', 'batchID': '1627', 'commission': '0.0', 'financing': '0.0', 'fullPrice': {'asks': [{'liquidity': '9975000', 'price': 1.18339}], 'bids': [{'liquidity': '10000000', 'price': 1.18326}], 'closeoutAsk': 1.18339, 'closeoutBid': 1.18326, 'type': 'PRICE'}, 'fullVWAP': 1.18326, 'gainQuoteHomeConversionFactor': '0.840850994445', 'guaranteedExecutionFee': '0.0', 'halfSpreadCost': '1.3732', 'id': '1628', 'instrument': 'EUR_USD', 'lossQuoteHomeConversionFactor': '0.849301758209', 'orderID': '1627', 'pl': '-4.0342', 'price': 1.18326, 'reason': 'MARKET_ORDER', 'requestID': '78757241552009237', 'time': '2020-08-13T12:07:20.586564454Z', 'tradesClosed': [{'financing': '0.0', 'guaranteedExecutionFee': '0.0', 'halfSpreadCost': '1.3732', 'price': 1.18326, 'realizedPL': '-4.0342', 'tradeID': '1626', 'units': '-25000.0'}],

352

|

Chapter 12: Execution and Deployment

'type': 'ORDER_FILL', 'units': '-25000.0', 'userID': 13834683} In [20]: print_details(order) Out[20]: ('2020-08-13T12:07:20.586', 'EUR_USD', '-25000.0', 1.18326, '-4.0342') In [21]: time.sleep(1)

Places a sell market order and prints the order object details Selects and shows the time, instrument, units, price, and pl details of the order

Limit Orders This chapter covers market orders as a type of base order only. With a market order, buying or selling a financial instrument happens at the price that is current when the order is placed. By contrast, a limit order, as the other main type of base order, allows the place‐ ment of an order with a minimum price or a maximum price. Only when the minimum/maximum price is reached is the order executed. Until that point, no transaction takes place.

Next, consider an example for the same combination of trades but this time with a stop loss (SL) order. An SL order is treated as a separate (limit) order. The following Python code places the orders and shows the details of the SL order object: In [22]: order = api.create_order('EUR_USD', units=25000, sl_distance=0.005, suppress=True, ret=True) In [23]: print_details(order) Out[23]: ('2020-08-13T12:07:21.740', 'EUR_USD', '25000.0', 1.18343, '0.0') In [24]: sl_order = api.get_transaction(tid=int(order['id']) + 1) In [25]: sl_order Out[25]: {'id': '1631', 'time': '2020-08-13T12:07:21.740825489Z', 'userID': 13834683, 'accountID': '101-004-13834683-001', 'batchID': '1629', 'requestID': '78757241556206373', 'type': 'STOP_LOSS_ORDER', 'tradeID': '1630', 'price': 1.17843, 'distance': '0.005', 'timeInForce': 'GTC', 'triggerCondition': 'DEFAULT', 'reason': 'ON_FILL'}

Order Execution

|

353

In [26]: (sl_order['time'], sl_order['type'], order['price'], sl_order['price'], sl_order['distance']) Out[26]: ('2020-08-13T12:07:21.740825489Z', 'STOP_LOSS_ORDER', 1.18343, 1.17843, '0.005') In [27]: time.sleep(1) In [28]: order = api.create_order('EUR_USD', units=-25000, suppress=True, ret=True) In [29]: print_details(order) Out[29]: ('2020-08-13T12:07:23.059', 'EUR_USD', '-25000.0', 1.18329, '-2.9725')

The SL distance is defined in currency units. Selects and shows the SL order object data. Selects and shows some relevant details of the two order objects. A trailing stop loss (TSL) order is handled in the same way. The only difference is that there is no fixed price attached to a TSL order: In [30]: order = api.create_order('EUR_USD', units=25000, tsl_distance=0.005, suppress=True, ret=True) In [31]: print_details(order) Out[31]: ('2020-08-13T12:07:23.204', 'EUR_USD', '25000.0', 1.18341, '0.0') In [32]: tsl_order = api.get_transaction(tid=int(order['id']) + 1) In [33]: tsl_order Out[33]: {'id': '1637', 'time': '2020-08-13T12:07:23.204457044Z', 'userID': 13834683, 'accountID': '101-004-13834683-001', 'batchID': '1635', 'requestID': '78757241564598562', 'type': 'TRAILING_STOP_LOSS_ORDER', 'tradeID': '1636', 'distance': '0.005', 'timeInForce': 'GTC', 'triggerCondition': 'DEFAULT', 'reason': 'ON_FILL'} In [34]: (tsl_order['time'][:-7], tsl_order['type'], order['price'], tsl_order['distance']) Out[34]: ('2020-08-13T12:07:23.204', 'TRAILING_STOP_LOSS_ORDER', 1.18341, '0.005')

354

|

Chapter 12: Execution and Deployment

In [35]: time.sleep(1) In [36]: order = api.create_order('EUR_USD', units=-25000, suppress=True, ret=True) In [37]: print_details(order) Out[37]: ('2020-08-13T12:07:24.551', 'EUR_USD', '-25000.0', 1.1833, '-2.3355') In [38]: time.sleep(1)

The TSL distance is defined in currency units. Selects and shows the TSL order object data. Selects and shows some relevant details of the two order objects. Finally, here is a take profit (TP) order. This order requires a fixed TP target price. Therefore, the following code uses the execution price from the previous order to define the TP price in relative terms. Beyond this small difference, the handling is again the same as before: In [39]: tp_price = round(order['price'] + 0.01, 4) tp_price Out[39]: 1.1933 In [40]: order = api.create_order('EUR_USD', units=25000, tp_price=tp_price, suppress=True, ret=True) In [41]: print_details(order) Out[41]: ('2020-08-13T12:07:25.712', 'EUR_USD', '25000.0', 1.18344, '0.0') In [42]: tp_order = api.get_transaction(tid=int(order['id']) + 1) In [43]: tp_order Out[43]: {'id': '1643', 'time': '2020-08-13T12:07:25.712531725Z', 'userID': 13834683, 'accountID': '101-004-13834683-001', 'batchID': '1641', 'requestID': '78757241572993078', 'type': 'TAKE_PROFIT_ORDER', 'tradeID': '1642', 'price': 1.1933, 'timeInForce': 'GTC', 'triggerCondition': 'DEFAULT', 'reason': 'ON_FILL'} In [44]: (tp_order['time'][:-7], tp_order['type'], order['price'], tp_order['price']) Out[44]: ('2020-08-13T12:07:25.712', 'TAKE_PROFIT_ORDER', 1.18344, 1.1933)

Order Execution

|

355

In [45]: time.sleep(1) In [46]: order = api.create_order('EUR_USD', units=-25000, suppress=True, ret=True) In [47]: print_details(order) Out[47]: ('2020-08-13T12:07:27.020', 'EUR_USD', '-25000.0', 1.18332, '-2.5478')

The TP target price is defined relative to the previous execution price. Selects and shows the TP order object data. Selects and shows some relevant details of the two order objects. The code so far only deals with transaction details of single orders. However, it is also of interest to have an overview of multiple historical transactions. To this end, the following method call provides overview data for all the main orders placed in this section, including P&L data: In [48]: api.print_transactions(tid=int(order['id']) - 22) 1626 | 2020-08-13T12:07:19.434407966Z | EUR_USD 1628 | 2020-08-13T12:07:20.586564454Z | EUR_USD 1630 | 2020-08-13T12:07:21.740825489Z | EUR_USD 1633 | 2020-08-13T12:07:23.059178023Z | EUR_USD 1636 | 2020-08-13T12:07:23.204457044Z | EUR_USD 1639 | 2020-08-13T12:07:24.551026466Z | EUR_USD 1642 | 2020-08-13T12:07:25.712531725Z | EUR_USD 1645 | 2020-08-13T12:07:27.020414342Z | EUR_USD

| | | | | | | |

25000.0 -25000.0 25000.0 -25000.0 25000.0 -25000.0 25000.0 -25000.0

| 0.0 | -4.0342 | 0.0 | -2.9725 | 0.0 | -2.3355 | 0.0 | -2.5478

Yet another method call provides a snapshot of the account details. The details shown are from an Oanda demo account that has been in use for quite some time for techni‐ cal testing purposes: In [49]: api.get_account_summary() Out[49]: {'id': '101-004-13834683-001', 'alias': 'Primary', 'currency': 'EUR', 'balance': '98541.4272', 'createdByUserID': 13834683, 'createdTime': '2020-03-19T06:08:14.363139403Z', 'guaranteedStopLossOrderMode': 'DISABLED', 'pl': '-1248.5543', 'resettablePL': '-1248.5543', 'resettablePLTime': '0', 'financing': '-210.0185', 'commission': '0.0', 'guaranteedExecutionFees': '0.0', 'marginRate': '0.0333', 'openTradeCount': 1, 'openPositionCount': 1,

356

| Chapter 12: Execution and Deployment

'pendingOrderCount': 0, 'hedgingEnabled': False, 'unrealizedPL': '941.9536', 'NAV': '99483.3808', 'marginUsed': '380.83', 'marginAvailable': '99107.2283', 'positionValue': '3808.3', 'marginCloseoutUnrealizedPL': '947.9546', 'marginCloseoutNAV': '99489.3818', 'marginCloseoutMarginUsed': '380.83', 'marginCloseoutPercent': '0.00191', 'marginCloseoutPositionValue': '3808.3', 'withdrawalLimit': '98541.4272', 'marginCallMarginUsed': '380.83', 'marginCallPercent': '0.00383', 'lastTransactionID': '1646'}

This concludes the discussion of the basics of executing orders with Oanda. All ele‐ ments are now together to support the deployment of a trading bot. The remainder of this chapter trains a trading bot on Oanda data and deploys it in automated fashion.

Trading Bot Chapter 11 shows in detail how to train a deep Q-learning trading bot and how to backtest it in vectorized and event-based fashion. This section now repeats selected core steps in this regard based on historical data from Oanda. “Oanda Environment” on page 369 provides a Python module that contains the environment class OandaEnv to work with Oanda data. It can be used in the same way as the Finance class from Chapter 11. The following Python code instantiates the learning environment object. During this step, the major data-related parameters driving the learning, validation, and testing are fixed. The OandaEnv class allows the inclusion of leverage, which is typical for FX and CFD trading. Leverage amplifies the realized returns, thereby increasing the profit potential but also the loss risks: In [50]: import oandaenv as oe In [51]: symbol = 'EUR_USD' In [52]: date = '2020-08-11' In [53]: features = [symbol, 'r', 's', 'm', 'v'] In [54]: %%time learn_env = oe.OandaEnv(symbol=symbol, start=f'{date} 08:00:00', end=f'{date} 13:00:00', granularity='S30',

Trading Bot

|

357

price='M', features=features, window=20, lags=3, leverage=20, min_accuracy=0.4, min_performance=0.85 ) CPU times: user 23.1 ms, sys: 2.86 ms, total: 25.9 ms Wall time: 26.8 ms In [55]: np.bincount(learn_env.data['d']) Out[55]: array([299, 281]) In [56]: learn_env.data.info()

DatetimeIndex: 580 entries, 2020-08-11 08:10:00 to 2020-08-11 12:59:30 Data columns (total 6 columns): # Column Non-Null Count Dtype --- ------------------- ----0 EUR_USD 580 non-null float64 1 r 580 non-null float64 2 s 580 non-null float64 3 m 580 non-null float64 4 v 580 non-null float64 5 d 580 non-null int64 dtypes: float64(5), int64(1) memory usage: 31.7 KB

Sets the granularity for the data to five seconds Sets the price type to mid prices Defines the set of features to be used Defines the window length for rolling statistics Specifies the number of lags Fixes the leverage Sets the required minimum accuracy Sets the required minimum performance In a next step, the validation environment is instantiated, relying on the parameters of the learning environment—apart from the time interval, for obvious reasons.

358

|

Chapter 12: Execution and Deployment

Figure 12-3 shows the closing prices of EUR/USD as used in the learning, validation, and test environments (from left to right): In [57]: valid_env = oe.OandaEnv(symbol=learn_env.symbol, start=f'{date} 13:00:00', end=f'{date} 14:00:00', granularity=learn_env.granularity, price=learn_env.price, features=learn_env.features, window=learn_env.window, lags=learn_env.lags, leverage=learn_env.leverage, min_accuracy=0, min_performance=0, mu=learn_env.mu, std=learn_env.std ) In [58]: valid_env.data.info()

DatetimeIndex: 100 entries, 2020-08-11 13:10:00 to 2020-08-11 13:59:30 Data columns (total 6 columns): # Column Non-Null Count Dtype --- ------------------- ----0 EUR_USD 100 non-null float64 1 r 100 non-null float64 2 s 100 non-null float64 3 m 100 non-null float64 4 v 100 non-null float64 5 d 100 non-null int64 dtypes: float64(5), int64(1) memory usage: 5.5 KB In [59]: test_env = oe.OandaEnv(symbol=learn_env.symbol, start=f'{date} 14:00:00', end=f'{date} 17:00:00', granularity=learn_env.granularity, price=learn_env.price, features=learn_env.features, window=learn_env.window, lags=learn_env.lags, leverage=learn_env.leverage, min_accuracy=0, min_performance=0, mu=learn_env.mu, std=learn_env.std ) In [60]: test_env.data.info()

DatetimeIndex: 340 entries, 2020-08-11 14:10:00 to 2020-08-11 16:59:30 Data columns (total 6 columns):

Trading Bot

|

359

# Column Non-Null Count Dtype --- ------------------- ----0 EUR_USD 340 non-null float64 1 r 340 non-null float64 2 s 340 non-null float64 3 m 340 non-null float64 4 v 340 non-null float64 5 d 340 non-null int64 dtypes: float64(5), int64(1) memory usage: 18.6 KB In [61]: ax = learn_env.data[learn_env.symbol].plot(figsize=(10, 6)) plt.axvline(learn_env.data.index[-1], ls='--') valid_env.data[learn_env.symbol].plot(ax=ax, style='-.') plt.axvline(valid_env.data.index[-1], ls='--') test_env.data[learn_env.symbol].plot(ax=ax, style='-.');

Figure 12-3. Historical 30-second bar closing prices for EUR/USD from Oanda (learning = left, validation = middle, testing = right) Based on the Oanda environment, the trading bot from Chapter 11 can be trained and validated. The following Python code performs this task and visualizes the per‐ formance results (see Figure 12-4): In [62]: import sys sys.path.append('../ch11/') In [63]: import tradingbot Using TensorFlow backend. In [64]: tradingbot.set_seeds(100) agent = tradingbot.TradingBot(24, 0.001, learn_env=learn_env, valid_env=valid_env)

360

|

Chapter 12: Execution and Deployment

In [65]: episodes = 31 In [66]: %time agent.learn(episodes) ======================================================================= episode: 5/31 | VALIDATION | treward: 97 | perf: 1.004 | eps: 0.96 ======================================================================= ======================================================================= episode: 10/31 | VALIDATION | treward: 97 | perf: 1.005 | eps: 0.91 ======================================================================= ======================================================================= episode: 15/31 | VALIDATION | treward: 97 | perf: 0.986 | eps: 0.87 ======================================================================= ======================================================================= episode: 20/31 | VALIDATION | treward: 97 | perf: 1.012 | eps: 0.83 ======================================================================= ======================================================================= episode: 25/31 | VALIDATION | treward: 97 | perf: 0.995 | eps: 0.79 ======================================================================= ======================================================================= episode: 30/31 | VALIDATION | treward: 97 | perf: 0.972 | eps: 0.75 ======================================================================= episode: 31/31 | treward: 16 | perf: 0.981 | av: 376.0 | max: 577 CPU times: user 22.1 s, sys: 1.17 s, total: 23.3 s Wall time: 20.1 s In [67]: tradingbot.plot_performance(agent)

Imports the tradingbot module from Chapter 11 Trains and validates the trading bot based on Oanda data Visualizes the performance results As discussed in the previous two chapters, the training and validation performances are just an indicator of the trading bot performance.

Trading Bot

|

361

Figure 12-4. Training and validation performance results of the trading bot for Oanda data The following code implements a vectorized backtest of the trading bot performance for the test environment—again with the same parameters as the learning environ‐ ment apart from the time interval used. The code makes use of the function backt est() as provided in the Python module presented in “Vectorized Backtesting” on page 372. The reported performance numbers include a leverage of 20. This holds true for both the gross performance of the passive benchmark investment and the trading bot over time, as shown in Figure 12-5: In [68]: import backtest as bt In [69]: env = test_env In [70]: bt.backtest(agent, env) In [71]: env.data['p'].iloc[env.lags:].value_counts() Out[71]: 1 263 -1 74 Name: p, dtype: int64 In [72]: sum(env.data['p'].iloc[env.lags:].diff() != 0) Out[72]: 25 In [73]: (env.data[['r', 's']].iloc[env.lags:] * env.leverage).sum( ).apply(np.exp) Out[73]: r 0.99966 s 1.05910 dtype: float64

362

| Chapter 12: Execution and Deployment

In [74]: (env.data[['r', 's']].iloc[env.lags:] * env.leverage).sum( ).apply(np.exp) - 1 Out[74]: r -0.00034 s 0.05910 dtype: float64 In [75]: (env.data[['r', 's']].iloc[env.lags:] * env.leverage).cumsum( ).apply(np.exp).plot(figsize=(10, 6));

Shows the total number of long and short positions Shows the number of trades required to implement the strategy Calculates the gross performance including leverage Calculates the net performance including leverage Visualizes the gross performance over time including leverage

Figure 12-5. Gross performance of the passive benchmark investment and the trading bot over time (including leverage)

Trading Bot

|

363

Simplified Backtesting The training and backtesting of the trading bot in this section hap‐ pen under assumptions that are not realistic. The trading strategy based on the 30-second bars might lead to a large number of trades over a short period of time. Assuming typical transaction costs (bid-ask spreads), such a strategy often is not economically viable. Longer bars or a strategy with fewer trades would be more realistic. However, to allow for a “quick” deployment demo in the next sec‐ tion, the training and backtest are implemented intentionally on the relatively short 30-second bars.

Deployment This section combines the major elements of the previous sections to deploy the trained trading bot in automated fashion. This is comparable to the point in time at which an AV is prepared to be deployed on the streets. The class OandaTradingBot presented in the following code inherits from the tpqoa class and adds some helper functions and the trading logic: In [76]: import tpqoa In [77]: class OandaTradingBot(tpqoa.tpqoa): def __init__(self, config_file, agent, granularity, units, verbose=True): super(OandaTradingBot, self).__init__(config_file) self.agent = agent self.symbol = self.agent.learn_env.symbol self.env = agent.learn_env self.window = self.env.window if granularity is None: self.granularity = agent.learn_env.granularity else: self.granularity = granularity self.units = units self.trades = 0 self.position = 0 self.tick_data = pd.DataFrame() self.min_length = (self.agent.learn_env.window + self.agent.learn_env.lags) self.pl = list() self.verbose = verbose def _prepare_data(self): self.data['r'] = np.log(self.data / self.data.shift(1)) self.data.dropna(inplace=True) self.data['s'] = self.data[self.symbol].rolling( self.window).mean() self.data['m'] = self.data['r'].rolling(self.window).mean() self.data['v'] = self.data['r'].rolling(self.window).std() self.data.dropna(inplace=True)

364

|

Chapter 12: Execution and Deployment

def

def

def

def

# self.data_ = (self.data - self.env.mu) / self.env.std self.data_ = (self.data - self.data.mean()) / self.data.std() _resample_data(self): self.data = self.tick_data.resample(self.granularity, label='right').last().ffill().iloc[:-1] self.data = pd.DataFrame(self.data['mid']) self.data.columns = [self.symbol,] self.data.index = self.data.index.tz_localize(None) _get_state(self): state = self.data_[self.env.features].iloc[-self.env.lags:] return np.reshape(state.values, [1, self.env.lags, self.env.n_features]) report_trade(self, time, side, order): self.trades += 1 pl = float(order['pl']) self.pl.append(pl) cpl = sum(self.pl) print('\n' + 75 * '=') print(f'{time} | *** GOING {side} ({self.trades}) ***') print(f'{time} | PROFIT/LOSS={pl:.2f} | CUMULATIVE={cpl:.2f}') print(75 * '=') if self.verbose: pprint(order) print(75 * '=') on_success(self, time, bid, ask): df = pd.DataFrame({'ask': ask, 'bid': bid, 'mid': (bid + ask) / 2}, index=[pd.Timestamp(time)]) self.tick_data = self.tick_data.append(df) self._resample_data() if len(self.data) > self.min_length: self.min_length += 1 self._prepare_data() state = self._get_state() prediction = np.argmax( self.agent.model.predict(state)[0, 0]) position = 1 if prediction == 1 else -1 if self.position in [0, -1] and position == 1: order = self.create_order(self.symbol, units=(1 - self.position) * self.units, suppress=True, ret=True) self.report_trade(time, 'LONG', order) self.position = 1 elif self.position in [0, 1] and position == -1: order = self.create_order(self.symbol, units=-(1 + self.position) * self.units, suppress=True, ret=True) self.report_trade(time, 'SHORT', order) self.position = -1

Deployment

|

365

For demonstration, the normalization is done with the real-time data statistics.1 Collects the tick data and resamples it to the required granularity. Returns the current state of the financial market. Collects the P&L figures for every trade. Calculates the cumulative P&L for all trades. Predicts the market direction and derives the signal (position). Checks whether the conditions for a long position (buy order) are met. Checks whether the conditions for a short position (sell order) are met. The application of this class is straightforward. First, an object is instantiated, provid‐ ing as the major input the trained trading bot agent from the previous section. Second, the streaming for the instrument to be traded needs to be started. Whenever new tick data arrives, the .on_success() method is called, which contains the main logic for both the processing of the tick data and the placement of trades. To speed things up a bit, the deployment example relies, as did the backtesting before, on 30-second bars. In a production context, when managing real money, a longer time interval might be the better choice—if only to reduce the number of trades and there‐ with the transaction costs: In [78]: otb = OandaTradingBot('../aiif.cfg', agent, '30s', 25000, verbose=False) In [79]: otb.tick_data.info()

Index: 0 entries Empty DataFrame In [80]: otb.stream_data(agent.learn_env.symbol, stop=1000) =========================================================================== 2020-08-13T12:19:32.320291893Z | *** GOING SHORT (1) *** 2020-08-13T12:19:32.320291893Z | PROFIT/LOSS=0.00 | CUMULATIVE=0.00 =========================================================================== =========================================================================== 2020-08-13T12:20:00.083985447Z | *** GOING LONG (2) *** 2020-08-13T12:20:00.083985447Z | PROFIT/LOSS=-6.80 | CUMULATIVE=-6.80

1 This little trick leads more quickly to trades in this particular context given the data used. For real deploy‐

ment, the statistics from the learning environment data are to be used for the normalization.

366

|

Chapter 12: Execution and Deployment

=========================================================================== =========================================================================== 2020-08-13T12:25:00.099901587Z | *** GOING SHORT (3) *** 2020-08-13T12:25:00.099901587Z | PROFIT/LOSS=-7.86 | CUMULATIVE=-14.66 =========================================================================== In [81]: print('\n' + 75 * '=') print('*** CLOSING OUT ***') order = otb.create_order(otb.symbol, units=-otb.position * otb.units, suppress=True, ret=True) otb.report_trade(otb.time, 'NEUTRAL', order) if otb.verbose: pprint(order) print(75 * '=') =========================================================================== *** CLOSING OUT *** =========================================================================== 2020-08-13T12:25:16.870357562Z | *** GOING NEUTRAL (4) *** 2020-08-13T12:25:16.870357562Z | PROFIT/LOSS=-3.19 | CUMULATIVE=-17.84 =========================================================================== ===========================================================================

Instantiates the OandaTradingBot object Starts the streaming of the real-time data and the trading Closes the final position after a certain number of ticks retrieved During the deployment, P&L figures are collected in the pl attribute, which is a list object. Once the trading has stopped, the P&L figures can be analyzed: In [82]: pl = np.array(otb.pl) In [83]: pl Out[83]: array([ 0. In [84]: pl.cumsum() Out[84]: array([ 0.

, -6.7959, -7.8594, -3.1862])

,

-6.7959, -14.6553, -17.8415])

P&L figures for all trades Cumulative P&L figures The simple deployment example illustrates that one can trade algorithmically and in automated fashion with a deep Q-learning trading bot in less than 100 lines of Python code. The major prerequisite is the trained trading bot (i.e., an instance of the

Deployment

|

367

tradingbot class). Many important aspects are intentionally left out here. For exam‐ ple, in a production environment, one would probably like to persist the data. One would also like to persist the order objects. Measures to make sure that the socket connection is still alive are also important (for example, by monitoring a heartbeat). Overall, security, reliability, logging, and monitoring are not really addressed. Some more details in this regard are provided in Hilpisch (2020).

The Python script in “Oanda Trading Bot” on page 373 presents a standalone exe‐ cutable version of the OandaTradingBot class. This represents a major step toward a more robust deployment option as compared to an interactive context such as Jupyter Notebook or Jupyter Lab. The script also includes functionality to add SL, TSL, or TP orders for the execution. The script expects a pickled version of the agent object in the current working directory. The following Python code pickles the object for later usage by the script: In [85]: import pickle In [86]: pickle.dump(agent, open('trading.bot', 'wb'))

Conclusions This chapter discusses central aspects of the execution of an algorithmic trading strat‐ egy and the deployment of a trading bot. The Oanda trading platform provides directly or indirectly with its v20 API all necessary capabilities to do the following: • Retrieve historical data • Train and backtest a trading bot (deep Q-learning agent) • Stream real-time data • Place market (and limit) orders • Make use of SL, TSL, and TP orders • Deploy a trading bot in an automated manner The prerequisites to implement all these steps are a demo account with Oanda, stan‐ dard hardware and software (open source only), and a stable internet connection. In other words, the barriers of entry to algorithmic trading for the purposes of exploit‐ ing economic inefficiencies are pretty low. This is in stark contrast, for example, to the training, design, and construction of AVs for deployment on public streets—the budgets of companies in the AV space run into the billions of dollars. In other words, the finance domain has distinctive advantages compared to other industries and domains with regard to the real-world deployment of AI agents, such as trading bots, as focused on in this and the previous chapter.

368

|

Chapter 12: Execution and Deployment

References Books and papers cited in this chapter: Hilpisch, Yves. 2020. Python for Algorithmic Trading: From Idea to Cloud Deployment. Sebastopol: O’Reilly. Litman, Todd. 2020. “Autonomous Vehicle Implementation Predictions.” Victoria Transport Policy Institute. https://oreil.ly/ds7YM.

Python Code This section contains code used and referenced in the main body of the chapter.

Oanda Environment The following is the Python module with the OandaEnv class to train a trading bot based on historical Oanda data: # # Finance Environment # # (c) Dr. Yves J. Hilpisch # Artificial Intelligence in Finance # # import math import tpqoa import random import numpy as np import pandas as pd

class observation_space: def __init__(self, n): self.shape = (n,)

class action_space: def __init__(self, n): self.n = n def sample(self): return random.randint(0, self.n - 1)

class OandaEnv: def __init__(self, symbol, start, end, granularity, price, features, window, lags, leverage=1, min_accuracy=0.5, min_performance=0.85,

References

|

369

mu=None, std=None): self.symbol = symbol self.start = start self.end = end self.granularity = granularity self.price = price self.api = tpqoa.tpqoa('../aiif.cfg') self.features = features self.n_features = len(features) self.window = window self.lags = lags self.leverage = leverage self.min_accuracy = min_accuracy self.min_performance = min_performance self.mu = mu self.std = std self.observation_space = observation_space(self.lags) self.action_space = action_space(2) self._get_data() self._prepare_data() def _get_data(self): ''' Method to retrieve data from Oanda. ''' self.fn = f'../../source/oanda/' self.fn += f'oanda_{self.symbol}_{self.start}_{self.end}_' self.fn += f'{self.granularity}_{self.price}.csv' self.fn = self.fn.replace(' ', '_').replace('-', '_').replace(':', '_') try: self.raw = pd.read_csv(self.fn, index_col=0, parse_dates=True) except: self.raw = self.api.get_history(self.symbol, self.start, self.end, self.granularity, self.price) self.raw.to_csv(self.fn) self.data = pd.DataFrame(self.raw['c']) self.data.columns = [self.symbol] def _prepare_data(self): ''' Method to prepare additional time series data (such as features data). ''' self.data['r'] = np.log(self.data / self.data.shift(1)) self.data.dropna(inplace=True) self.data['s'] = self.data[self.symbol].rolling(self.window).mean() self.data['m'] = self.data['r'].rolling(self.window).mean() self.data['v'] = self.data['r'].rolling(self.window).std() self.data.dropna(inplace=True) if self.mu is None: self.mu = self.data.mean() self.std = self.data.std() self.data_ = (self.data - self.mu) / self.std

370

|

Chapter 12: Execution and Deployment

self.data['d'] = np.where(self.data['r'] > 0, 1, 0) self.data['d'] = self.data['d'].astype(int) def _get_state(self): ''' Privat method that returns the state of the environment. ''' return self.data_[self.features].iloc[self.bar self.lags:self.bar].values def get_state(self, bar): ''' Method that returns the state of the environment. ''' return self.data_[self.features].iloc[bar - self.lags:bar].values def reset(self): ''' Method to reset the environment. ''' self.treward = 0 self.accuracy = 0 self.performance = 1 self.bar = self.lags state = self._get_state() return state def step(self, action): ''' Method to step the environment forwards. ''' correct = action == self.data['d'].iloc[self.bar] ret = self.data['r'].iloc[self.bar] * self.leverage reward_1 = 1 if correct else 0 reward_2 = abs(ret) if correct else -abs(ret) reward = reward_1 + reward_2 * self.leverage self.treward += reward_1 self.bar += 1 self.accuracy = self.treward / (self.bar - self.lags) self.performance *= math.exp(reward_2) if self.bar >= len(self.data): done = True elif reward_1 == 1: done = False elif (self.accuracy < self.min_accuracy and self.bar > self.lags + 15): done = True elif (self.performance < self.min_performance and self.bar > self.lags + 15): done = True else: done = False state = self._get_state() info = {} return state, reward, done, info

Python Code

|

371

Defines the path for the data file Defines the filename of the data file Reads the data if a corresponding data file exists Retrieves the data for the API if no such file exists Writes the data as a CSV file to disk Selects the column with the closing prices Renames the column to the instrument name (symbol) Reward for correct prediction Reward for the realized performance (return) Combined reward for prediction and performance

Vectorized Backtesting The following is the Python module with the helper function backtest to generate the data to do a vectorized backtest for a deep Q-learning trading bot. The code is also used in Chapter 11: # # Vectorized Backtesting of # Trading Bot (Financial Q-Learning Agent) # # (c) Dr. Yves J. Hilpisch # Artificial Intelligence in Finance # import numpy as np import pandas as pd pd.set_option('mode.chained_assignment', None) def reshape(s, env): return np.reshape(s, [1, env.lags, env.n_features]) def backtest(agent, env): done = False env.data['p'] = 0 state = env.reset() while not done: action = np.argmax( agent.model.predict(reshape(state, env))[0, 0]) position = 1 if action == 1 else -1

372

| Chapter 12: Execution and Deployment

env.data.loc[:, 'p'].iloc[env.bar] = position state, reward, done, info = env.step(action) env.data['s'] = env.data['p'] * env.data['r']

Oanda Trading Bot The following is the Python script with the OandaTradingBot class and code to deploy the class: # # Oanda Trading Bot # and Deployment Code # # (c) Dr. Yves J. Hilpisch # Artificial Intelligence in Finance # import sys import tpqoa import pickle import numpy as np import pandas as pd sys.path.append('../ch11/')

class OandaTradingBot(tpqoa.tpqoa): def __init__(self, config_file, agent, granularity, units, sl_distance=None, tsl_distance=None, tp_price=None, verbose=True): super(OandaTradingBot, self).__init__(config_file) self.agent = agent self.symbol = self.agent.learn_env.symbol self.env = agent.learn_env self.window = self.env.window if granularity is None: self.granularity = agent.learn_env.granularity else: self.granularity = granularity self.units = units self.sl_distance = sl_distance self.tsl_distance = tsl_distance self.tp_price = tp_price self.trades = 0 self.position = 0 self.tick_data = pd.DataFrame() self.min_length = (self.agent.learn_env.window + self.agent.learn_env.lags) self.pl = list() self.verbose = verbose def _prepare_data(self): ''' Prepares the (lagged) features data. '''

Python Code

|

373

def

def

def

def

374

|

self.data['r'] = np.log(self.data / self.data.shift(1)) self.data.dropna(inplace=True) self.data['s'] = self.data[self.symbol].rolling(self.window).mean() self.data['m'] = self.data['r'].rolling(self.window).mean() self.data['v'] = self.data['r'].rolling(self.window).std() self.data.dropna(inplace=True) self.data_ = (self.data - self.env.mu) / self.env.std _resample_data(self): ''' Resamples the data to the trading bar length. ''' self.data = self.tick_data.resample(self.granularity, label='right').last().ffill().iloc[:-1] self.data = pd.DataFrame(self.data['mid']) self.data.columns = [self.symbol,] self.data.index = self.data.index.tz_localize(None) _get_state(self): ''' Returns the (current) state of the financial market. ''' state = self.data_[self.env.features].iloc[-self.env.lags:] return np.reshape(state.values, [1, self.env.lags, self.env.n_features]) report_trade(self, time, side, order): ''' Reports trades and order details. ''' self.trades += 1 pl = float(order['pl']) self.pl.append(pl) cpl = sum(self.pl) print('\n' + 71 * '=') print(f'{time} | *** GOING {side} ({self.trades}) ***') print(f'{time} | PROFIT/LOSS={pl:.2f} | CUMULATIVE={cpl:.2f}') print(71 * '=') if self.verbose: pprint(order) print(71 * '=') on_success(self, time, bid, ask): ''' Contains the main trading logic. ''' df = pd.DataFrame({'ask': ask, 'bid': bid, 'mid': (bid + ask) / 2}, index=[pd.Timestamp(time)]) self.tick_data = self.tick_data.append(df) self._resample_data() if len(self.data) > self.min_length: self.min_length += 1 self._prepare_data() state = self._get_state() prediction = np.argmax(self.agent.model.predict(state)[0, 0]) position = 1 if prediction == 1 else -1 if self.position in [0, -1] and position == 1: order = self.create_order(self.symbol, units=(1 - self.position) * self.units, sl_distance=self.sl_distance, tsl_distance=self.tsl_distance,

Chapter 12: Execution and Deployment

tp_price=self.tp_price, suppress=True, ret=True) self.report_trade(time, 'LONG', order) self.position = 1 elif self.position in [0, 1] and position == -1: order = self.create_order(self.symbol, units=-(1 + self.position) * self.units, sl_distance=self.sl_distance, tsl_distance=self.tsl_distance, tp_price=self.tp_price, suppress=True, ret=True) self.report_trade(time, 'SHORT', order) self.position = -1

if __name__ == '__main__': agent = pickle.load(open('trading.bot', 'rb')) otb = OandaTradingBot('../aiif.cfg', agent, '5s', 25000, verbose=False) otb.stream_data(agent.learn_env.symbol, stop=1000) print('\n' + 71 * '=') print('*** CLOSING OUT ***') order = otb.create_order(otb.symbol, units=-otb.position * otb.units, suppress=True, ret=True) otb.report_trade(otb.time, 'NEUTRAL', order) if otb.verbose: pprint(order) print(71 * '=')

Python Code

|

375

PART V

Outlook

This part of the book serves as an epilogue. It provides an outlook on the conse‐ quences that the widespread adoption of AI in finance could have. It is concerned primarily, as is the rest of the book, with the trading domain to keep the discussion focused. This final part consists of two chapters: • Chapter 13 discusses aspects of AI-driven competition in the financial industry, such as new requirements for finance education or competitive scenarios that might arise. • Chapter 14 considers the prospect of a financial singularity and the emergence of an artificial financial intelligence—a trading bot consistently generating profits through algorithmic trading well beyond known human or institutional capabilities. This part is largely speculative, argues on a high level, and neglects many relevant and interesting details. It can serve, however, as a starting point for a more in-depth dis‐ cussion and analysis of the important topics addressed in it.

CHAPTER 13

AI-Based Competition

One high-stakes and extremely competitive environment in which AI systems operate today is the global financial market. —Nick Bostrom (2014) Financial services companies are becoming hooked on artificial intelligence, using it to automate menial tasks, analyze data, improve customer service and comply with regulations. —Nick Huber (2020)

This chapter addresses topics related to competition in the financial industry based on the systematic and strategic application of AI. “AI and Finance” on page 380 serves as a review and summary of the importance that AI might have for the future of finance. “Lack of Standardization” on page 382 argues that AI in finance is still at a nascent stage, making the implementation in many instances anything but straight‐ forward. This, on the other hand, leaves the competitive landscape wide open for financial players to secure competitive advantages through AI. The rise of AI in finance requires a rethinking and redesign of finance education and training. Today’s requirements cannot be met anymore by traditional finance curricula. “Fight for Resources” on page 385 discusses how financial institutions will fight for necessary resources to apply AI at a large scale in finance. As in many other areas, AI experts are often the bottleneck for which financial companies compete with technology companies, startups, and companies from other industries. “Market Impact” on page 386 explains how AI is both a major cause of and the only solution for the age of microscopic alpha—alpha that is, like gold nowadays, still to be found but only at small scales and in many cases to be mined only with industrial effort. “Competitive Scenarios” on page 387 discusses reasons for and against future scenarios for the financial industry characterized by a monopoly, oligopoly, or perfect competition. Finally, “Risks, Regulation, and Oversight” on page 388 has a brief look 379

at risks arising from AI in finance in general and major problems regulators and industry watchdogs are faced with.

AI and Finance This book primarily focuses on the use of AI in finance as applied to the prediction of financial time series. The goal is to discover statistical inefficiencies, situations in which the AI algorithm outperforms a baseline algorithm in predicting future market movements. Such statistical inefficiencies are the basis for economic inefficiencies. An economic inefficiency requires that there be a trading strategy that can exploit the statistical inefficiency in such a way that above-market returns are realized. In other words, there is a strategy—composed of the prediction algorithm and an execution algorithm—that generates alpha. There are, of course, many other areas in which AI algorithms can be applied to finance. Examples include the following: Credit scoring AI algorithms can be used to derive credit scores for potential debtors, thereby supporting credit decisions or even fully automating them. For example, Gol‐ bayani et al. (2020) apply a neural network–based approach to corporate credit ratings, whereas Babaev et al. (2019) use RNNs in the context of retail loan applications. Fraud detection AI algorithms can identify unusual patterns (for example, in transactions related to credit cards), thereby preventing fraud from remaining undetected or even from happening. Yousefi et al. (2019) provide a survey of the literature on the topic. Trade execution AI algorithms can learn how to best execute trades related to large blocks of shares, for example, thereby minimizing market impact and transaction costs. The paper by Ning et al. (2020) applies a double deep Q-learning algorithm to learn optimal trade execution policies. Derivatives hedging AI algorithms can be trained to optimally execute hedge transactions for single derivative instruments or portfolios composed of such instruments. The approach is often called deep hedging. Buehler et al. (2019) apply a reinforcement learning approach to implement deep hedging.

380

| Chapter 13: AI-Based Competition

Portfolio management AI algorithms can be used to compose and rebalance portfolios of financial instruments, say, in the context of long-term retirement savings plans. The recent book by López de Prado (2020) covers this topic in detail. Customer service AI algorithms can be used to process natural language, such as in the context of customer inquiries. Chat bots have therefore—like in many other industries— become quite popular in finance. The paper by Yu et al. (2020) discusses a finan‐ cial chat bot based on the popular bidirectional encoder representations from transformers (BERT) model, which has its origin within Google. All these application areas of AI in finance and others not listed here benefit from the programmatic availability of large amounts of relevant data. Why can we expect machine, deep, and reinforcement learning algorithms to perform better than tradi‐ tional methods from financial econometrics, such as OLS regression? There are a number of reasons: Big data While traditional statistical methods can often cope with larger data sets, they at the same time do not benefit too much performance-wise from increasing data volumes. On the other hand, neural network–based approaches often benefit tremendously when trained on larger data sets with regard to the relevant perfor‐ mance metrics. Instability Financial markets, in contrast to the physical world, do not follow constant laws. They are rather changing over time, sometimes in rapid fashion. AI algorithms can take this more easily into account by incrementally updating neural networks through online training, for example. Nonlinearity OLS regression, for example, assumes an inherent linear relationship between the features and the labels data. AI algorithms, such as neural networks, can in gen‐ eral more easily cope with nonlinear relationships. Nonnormality In financial econometrics, the assumption of normally distributed variables is ubiquitous. AI algorithms in general do not rely that much on such constraining assumptions. High dimensionality Traditional methods from financial econometrics have proven useful for prob‐ lems characterized by low dimensionality. Many problems in finance are cast into a context with a pretty low number of features (independent variables), such as

AI and Finance

|

381

one (CAPM) or maybe a few more. More advanced AI algorithms can easily deal with problems characterized by high dimensionality, taking into account even several hundred different features if required. Classification problems The toolbox of traditional econometrics is mainly based on approaches for esti‐ mation (regression) problems. These problems for sure form an important cate‐ gory in finance. However, classification problems are probably equally important. The toolbox of machine and deep learning provides a large menu of options for attacking classification problems. Unstructured data Traditional methods from financial econometrics can basically only deal with structured, numerical data. Machine and deep learning algorithms are able to also efficiently handle unstructured, text-based data. They can also handle both structured and unstructured data efficiently at the same time. Although the application of AI is in many parts of finance still at a nascent stage, some areas of application have proven to benefit tremendously from the paradigm shift to AI-first finance. It is therefore relatively safe to predict that machine, deep, and reinforcement learning algorithms will significantly reshape the way finance is approached and conducted in practice. Furthermore, AI has become the number one instrument in the pursuit of competitive advantages.

Lack of Standardization Traditional, normative finance (see Chapter 3) has reached a high degree of standard‐ ization. There are a number of textbooks available on different formal levels that basically teach and explain the very same theories and models. Two examples in this context are Copeland et al. (2005) and Jones (2012). The theories and models in turn rely in general on research papers published over the previous decades. When Black and Scholes (1973) and Merton (1973) published their theories and models to price European option contracts with a closed-form analytical formula, the financial industry immediately accepted the formula and the ideas behind it as a benchmark. Almost 50 years later, with many improved theories and models sug‐ gested in between, the Black-Scholes-Merton model and formula are still considered to be a benchmark, if not the benchmark, in option pricing. On the other hand, AI-first finance lacks a noticeable degree of standardization. There are numerous research papers published essentially on a daily basis (for exam‐ ple, on http://arxiv.org). Among other reasons, this is due to the fact that traditional publication venues with peer review are in general too slow to keep up with the fast pace in the field of AI. Researchers are keen to share their work with the public as fast as possible, often to not be outpaced by competing teams. A peer review process, 382

|

Chapter 13: AI-Based Competition

which also has its merits in terms of quality assurance, might take months, during which the research would not be published. In that sense, researchers more and more trust in the community to take care of the review while also ensuring early credit for their discoveries. Whereas it was not unusual decades ago that a new finance working paper circulated for years among experts before being peer reviewed and finally published, today’s research environment is characterized by much faster turnaround times and the will‐ ingness of researchers to put out work early that might not have been thoroughly reviewed and tested by others. As a consequence, there are hardly any standards or benchmark implementations available for the multitude of AI algorithms that are being applied to financial problems. These fast research publication cycles are in large part driven by the easy applicability of AI algorithms to financial data. Students, researchers, and practitioners hardly need more than a typical consumer notebook to apply the latest breakthroughs in AI to the financial domain. This is an advantage when compared to the constraints of econometric research some decades ago (in the form of limited data availability and limited compute power, for example). But it also often leads to the idea of “throwing as much spaghetti as possible at the wall” in the hope that some might stick. To some extent, the eagerness and urgency are also caused by investors, pushing investment managers to come up with new investment approaches at a faster pace. This often requires the dismissal of traditional research approaches in finance in favor of more practical approaches. As Lopéz de Prado (2018) puts it: Problem: Mathematical proofs can take years, decades, and centuries. No investor will wait that long. Solution: Use experimental math. Solve hard, intractable problems, not by proof but by experiment.

Overall, the lack of standardization provides ample opportunity for single financial players to exploit the advantages of AI-first finance in a competitive context. At the time of this writing in mid-2020, it feels like the race to leverage AI to revolutionize how finance is approached is moving at full speed. The remainder of this chapter addresses important aspects of AI-based competition beyond those of this and the previous section.

Education and Training Entering the field of finance and the financial industry happens quite often via a for‐ mal education in the field. Typical degrees have names such as the following: • Master of Finance • Master of Quantitative Finance Education and Training

|

383

• Master of Computational Finance • Master of Financial Engineering • Master of Quantitative Enterprise Risk Management Essentially, all such degrees today require the students to master at least one pro‐ gramming language, often Python, to address the data processing requirements of data-driven finance. In this regard, universities address the demand for these skills from the industry. Murray (2019) points out: The workforce will have to adapt as companies use artificial intelligence for more tasks. [T]here are opportunities for Master’s in Finance (MiF) graduates. The blend of tech‐ nological and financial knowledge is a sweet spot. Perhaps the highest demand comes from quantitative investors that use AI to trawl markets and colossal data sets to identify potential trades.

It is not only universities that adjust their curricula in finance-related degrees to include programming, data science, and AI. The companies themselves also invest heavily in training programs for new and existing staff to be ready for data-driven and AI-first finance. Noonan (2018) describes the large-scale training efforts of JPMorgan Chase, one of the largest banks in the world, as follows: JPMorgan Chase is putting hundreds of new investment bankers and asset managers through mandatory coding lessons, in a sign of Wall Street’s heightened need for tech‐ nology skills. With technology, from artificial intelligence trading to online lending platforms, shap‐ ing the future of banking, financial services groups are developing software to help them boost efficiency, create innovative products and fend off the threat from start-ups and tech giants. The coding training for this year’s juniors was based on Python programming, which will help them to analyze very large data sets and interpret unstructured data such as free language text. Next year, the asset management division will expand the manda‐ tory tech training to include data science concepts, machine learning and cloud computing.

In summary, more and more roles in the financial industry will require staff skilled in programming, basic and advanced data science concepts, machine learning, and other technical aspects, such as cloud computing. Universities and financial institu‐ tions on both the buy and sell sides react to this trend by adjusting their curricula and by investing heavily in training their workforces, respectively. In both cases, it is a matter of competing effectively—or even of staying relevant and being able to survive —in a financial landscape changed for good by the increasing importance of AI.

384

| Chapter 13: AI-Based Competition

Fight for Resources In the quest to make use of AI in a scalable, significant way in finance, players in the financial markets compete for the best resources. Four major resources are of para‐ mount importance: human resources, algorithms, data, and hardware. Probably the most important and, at the same time, scarcest resource is experts in AI in general and AI for finance in particular. In this regard, financial institutions com‐ pete with technology companies, financial technology (fintech) startups, and other groups for the best talent. Although banks are generally prepared to pay relatively high salaries to such experts, cultural aspects of technology companies and, for exam‐ ple, the promise of stock options in startups might make it difficult for them to attract top talent. Often, financial institutions resort to nurturing talent internally. Many algorithms and models in machine and deep learning can be considered stan‐ dard algorithms that are well researched, tested, and documented. In many instances, however, it is not clear from the outset how to apply them best in a financial context. This is where financial institutions invest heavily in research efforts. For many of the larger buy-side institutions, such as systematic hedge funds, investment and trading strategy research is at the very core of their business models. However, as Chapter 12 shows, deployment and production are of equal importance. Both strategy research and deployment are, of course, highly technical disciplines in this context. Algorithms without data are often worthless. Similarly, algorithms with “standard” data from typical data sources, such as exchanges or data service providers like Refi‐ nitiv or Bloomberg, might only be of limited value. This is due to the fact that such data is intensively analyzed by many, if not all, relevant players in the market, making it hard or even impossible to identify alpha-generating opportunities or similar com‐ petitive advantage. As a consequence, large buy-side institutions invest particularly heavily in getting access to alternative data (see “Data Availability” on page 104). How important alternative data is considered to be nowadays is reflected in invest‐ ments that buy-side players and other investors make in companies active in the field. For example, in 2018 a group of investment companies invested $95 million in the data group Enigma. Fortado (2018) describes the deal and its rationale as follows: Hedge funds, banks and venture capital firms are piling into investments in data com‐ panies in the hope of cashing in on a business they are using a lot more themselves. In recent years, there has been a proliferation of start-ups that trawl through reams of data and sell it to investment groups searching for an edge. The latest to attract investor interest is Enigma, a New York-based start-up that received funding from sources including quant giant Two Sigma, activist hedge fund Third Point and venture capital firms NEA and Glynn Capital in a $95m capital raising announced on Tuesday.

Fight for Resources

|

385

The fourth resource that financial institutions are competing for is the best hardware options to process big financial data, implement the algorithms based on traditional and alternative data sets, and thereby apply AI efficiently to finance. Recent years have seen tremendous innovation in hardware dedicated to making machine and deep learning efforts faster, more energy-efficient, and more cost-effective. While tra‐ ditional processors, such as CPUs, play a minor role in the field, specialized hardware such as GPUs by Nvidia or newer options such as TPUs by Google and IPUs by startup Graphcore have taken over in AI. The interest of financial institutions in new, specialized hardware is, for example, reflected in the research efforts of Citadel, one of the largest hedge funds and market makers, into IPUs. Its effort are documented in the comprehensive research report Jia et al. (2019), which illustrates the potential benefits of specialized hardware compared to alternative options. In the race to dominance in AI-first finance, financial institutions invest billions per year in talent, research, data, and hardware. Whereas large institutions seem well positioned to keep up with the pace in the field, smaller or medium-sized players will find it hard to comprehensively shift to an AI-first approach to their business.

Market Impact The increasing and now widespread usage of data science, machine learning, and deep learning algorithms in the financial industry without a doubt has an impact on financial markets, investment, and trading opportunities. As the many examples in this book illustrate, ML and DL methods are able to discover statistical inefficiencies and even economic inefficiencies that are not discoverable by traditional econometric methods, such as multivariate OLS regression. It is therefore to be assumed that new and better analysis methods make it harder to discover alpha-generating opportuni‐ ties and strategies. Comparing the current situation in financial markets with the one in gold mining, Lopéz de Prado (2018) describes the situation as follows: If a decade ago it was relatively common for an individual to discover macroscopic alpha (i.e., using simple mathematical tools like econometrics), currently the chances of that happening are quickly converging to zero. Individuals searching nowadays for macroscopic alpha, regardless of their experience or knowledge, are fighting overwhelming odds. The only true alpha left is microscopic, and finding it requires capital-intensive industrial methods. Just like with gold, microscopic alpha does not mean smaller overall profits. Microscopic alpha today is much more abundant than macroscopic alpha has ever been in history. There is a lot of money to be made, but you will need to use heavy ML tools.

Against this background, financial institutions almost seem to be required to embrace AI-first finance to not be left behind and eventually maybe even go out of business. This holds true not only in investing and trading, but in other areas as well.

386

|

Chapter 13: AI-Based Competition

While banks historically have nurtured long-term relationships with commercial and retail debtors and organically built their ability to make sound credit decisions, AI today levels the playing field and renders long-term relationships almost worthless. Therefore, new entrants in the field, such as fintech startups, relying on AI can often quickly grab market share from incumbents in a controlled, viable fashion. On the other hand, these developments incentivize incumbents to acquire and merge younger, innovative fintech startups to stay competitive.

Competitive Scenarios Looking forward, say, three to five years, how might the competitive landscape driven by AI-first finance look? Three scenarios are come to mind: Monopoly One financial institution reaches a dominant position through major, unmatched breakthroughs in applying AI to, say, algorithmic trading. This is, for example, the situation in internet searches, where Google has a global market share of about 90%. Oligopoly A smaller number of financial institutions are able to leverage AI-first finance to achieve leading positions. An oligopoly is, for example, also present in the hedge fund industry, in which a small number of large players dominate the field in terms of assets under management. Perfect competition All players in the financial markets benefit from advances in AI-first finance in similar fashion. No single player or group of players enjoys any competitive advantages compared to others. Technologically speaking, this is comparable to the situation in computer chess nowadays. A number of chess programs, running on standard hardware such as smartphones, are significantly better at playing chess than the current world champion (Magnus Carlsen at the time of this writing). It is hard to forecast which scenario is more likely. One can find arguments and describe possible paths for all three of them. For example, an argument for a monop‐ oly might be that a major breakthrough in algorithmic trading, for example, might lead to a fast, significant outperformance that helps accumulate more capital through reinvestments, as well as through new inflows. This in turn increases the available technology and research budget to protect the competitive advantage and attracts tal‐ ent that would be otherwise hard to win over. This whole cycle is self-reinforcing, and the example of Google in search—in connection with the core online advertising business—is a good one in this context.

Competitive Scenarios

|

387

Similarly, there are good reasons to anticipate an oligopoly. Currently, it is safe to assume that any large player in the trading business invests heavily in research and technology, with AI-related initiatives making up a significant part of the budget. As in other fields, say, recommender engines—think Amazon for books, Netflix for films, and Spotify for music—multiple companies might be able to reach similar breakthroughs at the same time. It is conceivable that the current leading systemic traders will be able to use AI-first finance to cement their leading positions. Finally, many technologies have become ubiquitous over the years. Strong chess pro‐ grams are only one example. Others might be maps and navigation systems or speech-based personal assistants. In a perfect competition scenario, a pretty large number of financial players would compete for minuscule alpha-creating opportuni‐ ties or even might be unable to generate returns distinguishable from plain market returns. At the same time, there are arguments against the three scenarios. The current land‐ scape has many players with equal means and incentives to leverage AI in finance. This makes it unlikely that only a single player will stand out and grab market share in investment management that is comparable to Google in search. At the same time, the number of small, medium-sized, and large players doing research in the field and the low barriers of entry in algorithmic trading make it unlikely that a select few can secure defendable competitive advantages. An argument against perfect competition is that, in the foreseeable future, algorithmic trading at a large scale requires a huge amount of capital and other resources. With regard to chess, DeepMind has shown with AlphaZero that there is always room for innovation and significant improve‐ ments, even if a field almost seems settled once and for all.

Risks, Regulation, and Oversight A simple Google search reveals that there is an active discourse going on about the risks of AI and its regulation in general, as well as in the financial services industry.1 This section cannot address all relevant aspects in this context, but it can address at least a few important ones. The following are some of the risks that the application of AI in finance introduces: Privacy Finance is a sensitive area with tight privacy laws. The use of AI at a large scale requires the use of—at least partly—private data from customers. This increases the risk that private data will be leaked or used in inappropriate ways. Such a risk

1 For a brief overview of these topics, see these articles by McKinsey: Confronting the risks of artificial intelli‐

gence and Derisking machine learning and artificial intelligence.

388

| Chapter 13: AI-Based Competition

obviously does not arise when publicly available data sources, such as for finan‐ cial time series data, are used. Bias

AI algorithms can easily learn biases that are inherent in data related, for exam‐ ple, to retail or corporate customers. Algorithms can only be as good and as objective in, say, judging the creditworthiness of a potential debtor as the data allows.2 Again, the problem of learning biases is not really a problem when work‐ ing with market data, for instance.

Inexplicability In many areas, it is important that decisions can be explained, sometimes in detail and in hindsight. This might be required by law or by investors wanting to understand why particular investment decisions have been taken. Take the exam‐ ple of investment and trading decisions. If an AI, based on a large neural network, decides algorithmically when and what to trade, it might be pretty diffi‐ cult and often impossible to explain in detail why the AI has traded the way it has. Researchers work actively and intensively on “explainable AI,” but there are obvious limits in this regard. Herding Since the stock market crash of 1987, it is clear what kind of risk herding in financial trading represents. In 1987, positive feedback trading in the context of large-scale synthetic replication programs for put options—in combination with stop loss orders—triggered the downward spiral. A similar herding effect could be observed in the 2008 hedge fund meltdown, which for the first time revealed the extent to which different hedge funds implement similar kinds of strategies. With regard to the flash crash in 2010, for which algorithmic trading was blamed by some, the evidence seems unclear. However, the more widespread use of AI in trading might pose a similar risk when more and more institutions apply similar approaches that have proven fruitful. Other areas are also prone to such an effect. Credit decision agents might learn the same biases based on different data sets and might make it impossible for certain groups or individuals to get credit at all. Vanishing alpha As has been argued before, the more widespread use of AI in finance at ever larger scales might make alpha in the markets disappear. Techniques must get better, and data may become “more alternative” to secure any competitive advan‐ tage. Chapter 14 takes a closer look at this in the context of a potential financial singularity.

2 For more on the problem of bias through AI and solutions to it, see Klein (2020).

Risks, Regulation, and Oversight

|

389

Beyond the typical risks of AI, AI introduces new risks specific to the financial domain. At the same time, it is difficult for lawmakers and regulators to keep up with the developments in the field and to comprehensively assess individual and systemic risks arising from AI-first finance. There are several reasons for this: Know-how Lawmakers and regulators need to acquire, like the financial players themselves, new know-how related to AI in finance. In this respect, they compete with the big financial institutions and technology companies that are known to pay salar‐ ies well above the possibilities of lawmakers and regulators. Insufficient data In many application areas, there is simply little or even no data available that watchdogs can use to judge the real impact of AI. In some instances, it might not even be known whether AI plays a role or not. And even if it is known and data might be available, it might be hard to separate the impact of AI from the impact of other relevant factors. Little transparency While almost all financial institutions try to make use of AI to secure or gain competitive advantages, it is hardly ever transparent what a single institution does in this regard and how exactly it is implemented and used. Many treat their effort in this context as intellectual property and their own “secret sauce.” Model validation Model validation is a central risk management and regulatory tool in many financial areas. Take the simple example of the pricing of a European option based on the Black-Scholes-Merton (1973) option pricing model. The prices that a specific implementation of the model generates can be validated, for example, by the use of the Cox et al. (1979) binomial option pricing model—and vice versa. This is often quite different with AI algorithms. There is hardly ever a model that, based on a parsimonious set of parameters, can validate the outputs of a complex AI algorithm. Reproducibility might be, however, an attainable goal (that is, the option to have third parties verify the outputs based on an exact rep‐ lication of all steps involved). But this in turn would require the third party, say, a regulator or an auditor, to have access to the same data, an infrastructure as pow‐ erful as the one used by the financial institution, and so on. For larger AI efforts, this seems simply unrealistic. Hard to regulate Back to the option pricing example, a regulator can specify that both the BlackScholes-Merton (1973) and the Cox et al. (1979) option pricing models are acceptable for the pricing of European options. Even when lawmakers and regu‐ lators specify that both support vector machine (SVM) algorithms and neural

390

|

Chapter 13: AI-Based Competition

networks are “acceptable algorithms,” this leaves open how these algorithms are trained, used, and so on. It is difficult to be more specific in this context. For example, should a regulator limit the number of hidden layers and/or hidden units in a neural network? What about the software packages to be used? The list of hard questions seems endless. Therefore, only general rules will be formulated. Technology companies and financial institutions alike usually prefer a more lax approach to AI regulation—for often obvious reasons. In Bradshaw (2019), Google CEO Sundar Pichai speaks of “smart” regulation and asks for an approach that differ‐ entiates between different industries: Google’s chief executive has warned politicians against knee-jerk regulation of artificial intelligence, arguing that existing rules may be sufficient to govern the new technology. Sundar Pichai said that AI required “smart regulation” that balanced innovation with protecting citizens…."It is such a broad cross-cutting technology, so it’s important to look at [regulation] more in certain vertical situations,” Mr Pichai said.

On the other hand, there are popular proponents of a more stringent regulation of AI, such as Elon Musk in Matyus (2020): “Mark my words,” Musk warned. “A.I. is far more dangerous than nukes. So why do we have no regulatory oversight?”

The risks from AI in finance are manifold, as are the problems faced by lawmakers and regulators. Nevertheless, it is safe to predict that tighter regulation and oversight addressing AI in finance specifically is certainly to come in many jurisdictions.

Conclusions This chapter addresses aspects of using AI to compete in the financial industry. The benefits are clear in many application areas. However, so far hardly any standards have been established, and the field seems still wide open for players to strive for competitive advantages. Because new technologies and approaches from data science, machine learning, deep learning, and more generally AI infiltrate almost any finan‐ cial discipline, education and training in finance must take this into account. Many master’s programs have already adjusted their curricula, while big financial institu‐ tions invest heavily in training incoming and existing staff in the required skills. Beyond human resources, financial institutions also compete for other resources in the field, such as alternative data. In the financial markets, AI-powered investment and trading make it harder to identify sustainable alpha opportunities. On the other hand, with traditional econometric methods it might be impossible today to identify and mine microscopic alpha. It is difficult to predict a competitive end scenario for the financial industry at a point when AI has taken over. Scenarios ranging from a monopoly to an oligopoly to per‐ fect competition seem still reasonable. Chapter 14 revisits this topic. AI-first finance Conclusions

|

391

confronts researchers, practitioners, and regulators with new risks and new chal‐ lenges in addressing these risks appropriately. One such risk, playing a prominent role in many discussions, is the black box characteristic of many AI algorithms. Such a risk usually can only be mitigated to some extent with today’s state-of-the-art explainable AI.

References Books, papers, and articles cited in this chapter: Babaev, Dmitrii et al. 2019. “E.T.-RNN: Applying Deep Learning to Credit Loan Applications.” https://oreil.ly/ZK5G8. Black, Fischer, and Myron Scholes. 1973. “The Pricing of Options and Corporate Lia‐ bilities.” Journal of Political Economy 81 (3): 638–659. Bradshaw, Tim. 2019. “Google chief Sundar Pichai warns against rushing into AI reg‐ ulation.” Financial Times, September 20, 2019. Bostrom, Nick. 2014. Superintelligence: Paths, Dangers, Strategies. Oxford: Oxford University Press. Buehler, Hans et al. 2019. “Deep Hedging: Hedging Derivatives Under Generic Mar‐ ket Frictions Using Reinforcement Learning.” Finance Institute Research Paper No. 19-80. https://oreil.ly/_oDaO. Copeland, Thomas, Fred Weston, and Kuldeep Shastri. 2005. Financial Theory and Corporate Policy. 4th ed. Boston: Pearson. Cox, John, Stephen Ross, and Mark Rubinstein. 1979. “Option Pricing: A Simplified Approach.” Journal of Financial Economics 7, (3): 229–263. Fortado, Lindsay. 2018. “Data specialist Enigma reels in investment group cash.” Financial Times, September 18, 2018. Golbayani, Parisa, Dan Wang, and Ionut Florescu. 2020. “Application of Deep Neural Networks to Assess Corporate Credit Rating.” https://oreil.ly/U3eXF. Huber, Nick. 2020. “AI ‘Only Scratching the Surface’ of Potential in Financial Serv‐ ices.” Financial Times, July 1, 2020. Jia, Zhe et al. 2019. “Dissecting the Graphcore IPU Architecture via Microbe‐ nchmarking.” https://oreil.ly/3ZgTO. Jones, Charles P. 2012. Investments: Analysis and Management. 12th ed. Hoboken: John Wiley & Sons. Klein, Aaron. 2020. “Reducing Bias in AI-based Financial Services.” The Brookings Institution Report, July 10, 2020, https://bit.ly/aiif_bias.

392

| Chapter 13: AI-Based Competition

López de Prado, Marcos. 2018. Advances in Financial Machine Learning. Hoboken: Wiley Finance. ⸻. 2020. Machine Learning for Asset Managers. Cambridge: Cambridge Univer‐ sity Press. Matyus, Allison. 2020. “Elon Musk Warns that All A.I. Must Be Regulated, Even at Tesla.” Digital Trends, February 18, 2020. https://oreil.ly/JmAKZ. Merton, Robert C. 1973. “Theory of Rational Option Pricing.” Bell Journal of Econom‐ ics and Management Science 4 (Spring): 141–183. Murray, Seb. 2019. “Graduates with Tech and Finance Skills in High Demand.” Finan‐ cial Times, June 17, 2019. Ning, Brian, Franco Ho Ting Lin, and Sebastian Jaimungal. 2020. “Double Deep QLearning for Optimal Execution.” https://oreil.ly/BSBNV. Noonan, Laura. 2018. “JPMorgan’s requirement for new staff: coding lessons.” Finan‐ cial Times, October 8, 2018. Yousefi, Niloofar, Marie Alaghband, and Ivan Garibay. 2019. “A Comprehensive Sur‐ vey on Machine Learning Techniques and User Authentication Approaches for Credit Card Fraud Detection.” https://oreil.ly/fFjAJ. Yu, Shi, Yuxin Chen, and Hussain Zaidi. 2020. “AVA: A Financial Service Chatbot based on Deep Bidirectional Transformers.” https://oreil.ly/2NVNH.

References

|

393

CHAPTER 14

Financial Singularity

We find ourselves in a thicket of strategic complexity, surrounded by a dense mist of uncertainty. —Nick Bostrom (2014) “Most trading and investment roles will disappear and over time, probably most roles that require human services will be automated,” says Mr Skinner. “What you will end up with is banks that are run primarily by managers and machines. The managers decide what the machines need to do, and the machines do the job.” —Nick Huber (2020)

Can AI-based competition in the financial industry lead to a financial singularity? This is the main question that this final chapter discusses. It starts with “Notions and Definitions” on page 396, which defines expressions such as financial singularity and artificial financial intelligence (AFI). “What Is at Stake?” on page 396 illustrates what, in terms of potential wealth accumulation, is at stake in the race for an AFI. “Paths to Financial Singularity” on page 400 considers, against the background of Chapter 2, paths that might lead to an AFI. “Orthogonal Skills and Resources” on page 401 argues that there are a number of resources that are instrumental and orthogonal to the goal of creating an AFI. Anybody involved in the race for an AFI will compete for these resources. Finally, “Star Trek or Star Wars” on page 403 considers whether an AFI, as discussed in this chapter, will benefit only a few people or humanity as a whole.

395

Notions and Definitions The expression financial singularity dates back at least to the 2015 blog post by Shiller. In this post, Shiller writes: Will alpha eventually go to zero for every imaginable investment strategy? More funda‐ mentally, is the day approaching when, thanks to so many smart people and smarter computers, financial markets really do become perfect, and we can just sit back, relax, and assume that all assets are priced correctly? This imagined state of affairs might be called the financial singularity, analogous to the hypothetical future technological singularity, when computers replace human intelli‐ gence. The financial singularity implies that all investment decisions would be better left to a computer program, because the experts with their algorithms have figured out what drives market outcomes and reduced it to a seamless system.

A bit more generally, one could define the financial singularity as the point in time from which computers and algorithms begin to take over control of finance and the whole financial industry, including banks, asset managers, exchanges, and so on, with humans taking a back seat as managers, supervisors, and controllers, if anything. On the other hand, one could define the financial singularity—in the spirit of this book’s focus—as the point in time from which a trading bot exists that shows a consis‐ tent capability to predict movements in financial markets at superhuman and superinstitutional levels, as well as with unprecedented accuracy. In that sense, such a trading bot would be characterized as an artificial narrow intelligence (ANI) instead of an artificial general intelligence (AGI) or superintelligence (see Chapter 2). It can be assumed that it is much easier to build such an AFI in the form of a trading bot than an AGI or even a superintelligence. This holds true for AlphaZero in the same way, as it is easier to build an AI agent that is superior to any human being or any other agent in playing the game of Go. Therefore, even if it is not yet clear whether there will ever be an AI agent that qualifies as an AGI or superintelligence, it is in any case much more likely that a trading bot will emerge that qualifies as an ANI or AFI. In what follows, the focus lies on a trading bot that qualifies as an AFI to keep the discussion as specific as possible and embedded in the context of this book.

What Is at Stake? The pursuit of an AFI might be challenging and exciting in and of itself. However, as is usual in finance, not too many initiatives are driven by altruistic motives; rather, most are driven by the financial incentives (that is, hard cash). But what exactly is at stake in the race to build an AFI? This cannot be answered with certainty or general‐ ity, but some simple calculations can shed light on the question.

396

|

Chapter 14: Financial Singularity

To understand how valuable it is to have an AFI as compared to inferior trading strategies, consider the following benchmarks: Bull strategy A trading strategy that goes long only on a financial instrument in the expecta‐ tion of rising prices. Random strategy A trading strategy that chooses a long or short position randomly for a given financial instrument. Bear strategy A trading strategy that goes short only on a financial instrument in the expecta‐ tion of falling prices. These benchmark strategies shall be compared to AFIs with the following success characteristics: X% top The AFI gets the top X% up and down movements correct, with the remaining market movements being predicted randomly. X% AFI The AFI gets X% of all randomly chosen market movements correct, with the remaining market movements being predicted randomly. The following Python code imports the known time series data set with EOD data for a number of financial instruments. The examples to follow rely on five years’ worth of EOD data for a single financial instrument: In [1]: import random import numpy as np import pandas as pd from pylab import plt, mpl plt.style.use('seaborn') mpl.rcParams['savefig.dpi'] = 300 mpl.rcParams['font.family'] = 'serif' In [2]: url = 'https://hilpisch.com/aiif_eikon_eod_data.csv' In [3]: raw = pd.read_csv(url, index_col=0, parse_dates=True) In [4]: symbol = 'EUR=' In [5]: raw['bull'] = np.log(raw[symbol] / raw[symbol].shift(1)) In [6]: data = pd.DataFrame(raw['bull']).loc['2015-01-01':] In [7]: data.dropna(inplace=True)

What Is at Stake?

|

397

In [8]: data.info()

DatetimeIndex: 1305 entries, 2015-01-01 to 2020-01-01 Data columns (total 1 columns): # Column Non-Null Count Dtype --- ------ -------------- ----0 bull 1305 non-null float64 dtypes: float64(1) memory usage: 20.4 KB

The bull benchmark returns (long only) With the bull strategy being already defined by the log returns of the base financial instrument, the following Python code specifies the other two benchmark strategies and derives the performances for the AFI strategies. In this context, a number of AFI strategies are considered to illustrate the impact of improvements in the accuracy of the AFI’s predictions: In [9]: np.random.seed(100) In [10]: data['random'] = np.random.choice([-1, 1], len(data)) * data['bull'] In [11]: data['bear'] = -data['bull'] In [12]: def top(t): top = pd.DataFrame(data['bull']) top.columns = ['top'] top = top.sort_values('top') n = int(len(data) * t) top['top'].iloc[:n] = abs(top['top'].iloc[:n]) top['top'].iloc[n:] = abs(top['top'].iloc[n:]) top['top'].iloc[n:-n] = np.random.choice([-1, 1], len(top['top'].iloc[n:-n])) * top['top'].iloc[n:-n] data[f'{int(t * 100)}_top'] = top.sort_index() In [13]: for t in [0.1, 0.15]: top(t) In [14]: def afi(ratio): correct = np.random.binomial(1, ratio, len(data)) random = np.random.choice([-1, 1], len(data)) strat = np.where(correct, abs(data['bull']), random * data['bull']) data[f'{int(ratio * 100)}_afi'] = strat In [15]: for ratio in [0.51, 0.6, 0.75, 0.9]: afi(ratio)

The random benchmark returns The bear benchmark returns (short only)

398

|

Chapter 14: Financial Singularity

The X% top strategy returns The X% AFI strategy returns Using the standard vectorized backtesting approach, as introduced in Chapter 10 (neglecting transaction costs), it becomes clear what significant increases in the pre‐ diction accuracy imply in financial terms. Consider the “90% AFI,” which is not perfect in its predictions but rather lacks any edge in 10% of all cases. The assumed 90% accuracy leads to a gross performance that over five years returns almost 100 times the invested capital (before transaction costs). With 75% accuracy, the AFI would still return almost 50 times the invested capital (see Figure 14-1). This excludes leverage, which can easily be added in an almost risk-less fashion in the presence of such prediction accuracies: In [16]: data.head() Out[16]: bull Date 2015-01-01 0.000413 2015-01-02 -0.008464 2015-01-05 -0.005767 2015-01-06 -0.003611 2015-01-07 -0.004299

random

bear

10_top

-0.000413 0.008464 -0.005767 -0.003611 -0.004299

-0.000413 0.008464 0.005767 0.003611 0.004299

0.000413 0.008464 -0.005767 -0.003611 0.004299

60_afi

75_afi

90_afi

0.000413 0.008464 0.005767 0.003611 0.004299

0.000413 0.008464 -0.005767 0.003611 0.004299

0.000413 0.008464 0.005767 0.003611 0.004299

Date 2015-01-01 2015-01-02 2015-01-05 2015-01-06 2015-01-07

15_top

51_afi

\

-0.000413 0.000413 0.008464 0.008464 0.005767 -0.005767 0.003611 0.003611 0.004299 0.004299

In [17]: data.sum().apply(np.exp) Out[17]: bull 0.926676 random 1.097137 bear 1.079126 10_top 9.815383 15_top 21.275448 51_afi 12.272497 60_afi 22.103642 75_afi 49.227314 90_afi 98.176658 dtype: float64 In [18]: data.cumsum().apply(np.exp).plot(figsize=(10, 6));

What Is at Stake?

|

399

Figure 14-1. Gross performance of benchmark and theoretical AFI strategies over time The analyses show that quite a lot is at stake, although several simplifying assump‐ tions are of course made. Time plays an important role in this context. Reimplementing the same analyses over a 10-year period makes the numbers even more impressive—almost unimaginable in a trading context. As the following output illus‐ trates for “90% AFI,” the gross return would be more than 16,000 times the invested capital (before transaction costs). The effect of compounding and reinvesting is tremendous: bull 0.782657 random 0.800253 bear 1.277698 10_top 165.066583 15_top 1026.275100 51_afi 206.639897 60_afi 691.751006 75_afi 2947.811043 90_afi 16581.526533 dtype: float64

Paths to Financial Singularity The emergence of an AFI would be a rather specific event in a rather specified envi‐ ronment. It is, for example, not necessary to emulate the brain of a human being since AGI or superintelligence is not the major goal. Given that there is no human being who seems to be consistently superior in trading in the financial markets compared to everybody else, it might even be a dead end street, trying to emulate a human brain to arrive at an AFI. There is also no need to worry about embodiment. An AFI can

400

|

Chapter 14: Financial Singularity

live as software only on an appropriate infrastructure connecting to the required data and trading APIs. On the other hand, AI seems to be a promising path to an AFI because of the very nature of the problem: take as input large amounts of financial and other data and generate predictions about the future direction of a price movement. This is exactly what the algorithms presented and applied in this book are all about—in particular those that fall in the supervised and reinforcement learning categories. Another option might be a hybrid of human and machine intelligence. Whereas machines have supported human traders for decades, in many instances the roles have changed. Humans support the machines in trading by providing the ideal envi‐ ronment and up-to-date data, intervening only in extreme situations, and so forth. In many cases, the machines are already completely autonomous in their algorithmic trading decisions. Or as Jim Simons—founder of Renaissance Technologies, one of the most successful and secretive systematic trading hedge funds—puts it: “The only rule is we never override the computer.” While it is quite unclear which paths might lead to superintelligence, from today’s perspective, it seems most likely that AI might pave the way to the financial singular‐ ity and an AFI.

Orthogonal Skills and Resources Chapter 13 discusses the competition for resources in the context of AI-based compe‐ tition in the financial industry. The four major resources in this context are human resources (experts), algorithms, and software, financial and alternative data, and high-performance hardware. We can add a fifth resource in this context in the form of the capital needed to acquire the other resources. According to the orthogonality hypothesis, it is sensible and even imperative to acquire such orthogonal skills and resources, which are instrumental no matter how exactly an AFI will be achieved. Financial institutions taking part in the race to build an AFI will try to acquire as many high-quality resources as they can afford and jus‐ tify to position themselves as advantageously as possible for the point in time at which the, or at least one, path to an AFI becomes clear. In a world driven by AI-first finance, such behavior and positioning might make the difference between thriving, merely surviving, or leaving the market. It cannot be excluded that progress could be made much faster than expected. When Nick Bos‐ trom in 2014 predicted that it might take 10 years until an AI could beat the world champion in the game of Go, basically nobody expected that it would happen only two years later. The major driver has been breakthroughs in the application of reinforcement learning to such games, from which other applications still benefit today. Such unforeseen breakthroughs cannot be excluded in finance either. Orthogonal Skills and Resources

|

401

Scenarios Before and After It is safe to assume that every major financial institution around the world and many other nonfinancial entities currently do research and have practical experience with AI applied to finance. However, not all players in the financial industry are positioned equally well to arrive at a trading AFI first. Some, like banks, are rather restricted by regulatory requirements. Others simply follow different business models, such as exchanges. Others, like certain assets managers, focus on providing low-cost, com‐ moditized investment products, such as ETFs, that mimic the performance of broader market indices. In other words, generating alpha is not the major goal for every financial institution. From an outside perspective, larger hedge funds therefore seem best positioned to make the most out of AI-first finance and AI-powered algorithmic trading. In gen‐ eral, they already have a lot of the required resources important in this field: talented and well-educated people, experience with trading algorithms, and almost limitless access to traditional and alternative data sources, as well as a scalable, professional trading infrastructure. If something is missing, large technology budgets ensure quick and targeted investments. It is not clear whether there will be one AFI first with others coming later or if several AFIs may emerge at the same time. If several AFIs are present, one could speak of a multipolar or oligopolistic scenario. The AFIs would probably mostly compete against each other, with “non-AFI” players being sidelined. The sponsors of the single projects would strive to gain advantages, however small, because this might allow one AFI to take over completely and finally become a singleton or monopoly. It is also conceivable that a “winner take all” scenario might prevail from the start. In such a scenario, a single AFI emerges and is able to quickly reach a level of domi‐ nance in financial trading that cannot be matched by any other competitor. This could be for several reasons. One reason might be that the first AFI generates returns so impressive that the assets under management swell at a tremendous pace, leading to ever higher budgets that in turn allows it to acquire ever more relevant resources. Another reason might be that the first AFI quickly reaches a size at which its actions can have a market impact—with the ability to manipulate market prices, for instance —such that it becomes the major, or even the only, driving force in financial markets. Regulation could in theory prevent an AFI from becoming too big or gaining too much market power. The major questions would be if such laws are enforceable in practice and how exactly they would need to be designed to have their desired effects.

402

|

Chapter 14: Financial Singularity

Star Trek or Star Wars The financial industry, for many people, represents the purest form of capitalism: the industry in which greed drives everything. It has been and is for sure a highly competitive industry, no arguing about that. Trading and investment management in particular are often symbolized by billionaire managers and owners who are willing to bet big and go head-to-head with their rivals in order to land the next mega deal or trade. The advent of AI provides ambitious managers with a rich tool set to push the competition to the next level, as discussed in Chapter 13. However, the question is whether AI-first finance, potentially culminating in an AFI, will lead to financial utopia or dystopia. The systematic, infallible accumulation of wealth could theoretically serve only a few people, or it could potentially serve humanity. Unfortunately, it is to be assumed that only the sponsors of the project leading to an AFI will directly benefit from the type of AFI imagined in this chapter. This is because such an AFI would only generate profits by trading in the financial markets and not by inventing new products, solving important problems, or growing businesses and industries. In other words, an AFI that merely trades in the financial markets to generate profits is taking part in a zero-sum game and does not directly increase the distributable wealth. One could argue that, for example, pension funds investing in a fund managed by the AFI would also benefit from its exceptional returns. But this would again only benefit a certain group and not humanity as a whole. It would also be in question whether the sponsors of a successful AFI project would be willing to open up to outside investors. A good example in this regard is the Medallion fund, managed by Renais‐ sance Technologies and one of the best-performing investment vehicles in history. Renaissance closed Medallion, which is essentially run exclusively by machines, to outside investors in 1993. Its stellar performance for sure would have attracted large amounts of additional assets. However, specific considerations, such as the capacity of certain strategies, play a role in this context, and similar considerations might also apply to an AFI. Therefore, whereas one could expect a superintelligence to help overcome fundamen‐ tal problems faced by humanity as a whole—serious diseases, environmental prob‐ lems, unknown threats from outer space, and so forth—an AFI more probably leads to more inequality and fiercer competition in the markets. Instead of a Star Trek–like world, characterized by equality and inexhaustible resources, it cannot be excluded that an AFI might rather lead to a Star Wars–like world, characterized by intensive trade wars and fights over available resources. At the time of this writing, global trade wars, such as the one between the US and China, seem more intense than ever, and technology and AI are important battlegrounds.

Star Trek or Star Wars

|

403

Conclusions This chapter takes a high-level perspective and discusses the concepts of financial sin‐ gularity and artificial financial intelligence. AFI is an ANI that would lack many of the capabilities and characteristics of a superintelligence. An AFI could rather be compared to AlphaZero, which is an ANI for playing board games such as chess or Go. An AFI would excel at the game of trading financial instruments. Of course, in financial trading, a lot more is at stake compared to playing board games. Similar to AlphaZero, AI is more likely to pave the way to an AFI as compared to alternative paths, such as the emulation of the human brain. Even if the path is not yet fully visible, and although one cannot know for sure how far single projects have pro‐ gressed already, there are a number of instrumental resources that are important no matter which path will prevail: experts, algorithms, data, hardware, and capital. Large and successful hedge funds seem best positioned to win the race for an AFI. Even if it might prove impossible to create an AFI as sketched in this chapter, the sys‐ tematic introduction of AI to finance will certainly spur innovation and in many cases intensify competition in the industry. Rather than being a fad, AI is a trend that finally will lead to a paradigm shift in the industry.

References Books and papers cited in this chapter: Bostrom, Nick. 2014. Superintelligence: Paths, Dangers, Strategies. Oxford: Oxford University Press Huber, Nick. 2020. “AI ‘Only Scratching the Surface’ of Potential in Financial Serv‐ ices.” Financial Times, July 1, 2020. Shiller, Robert. 2015. “The Mirage of the Financial Singularity.” Yale Insights (blog). https://oreil.ly/cnWBh.

404

| Chapter 14: Financial Singularity

PART VI

Appendixes

This part serves as an appendix and presents additional material to support the con‐ tents, code, and examples presented in the other parts of this book. This part consists of three appendixes: • Appendix A covers fundamental notions related to neural networks, such as tensor operations. • Appendix B presents classes implementing simple and shallow neural networks from scratch. • Appendix C illustrates the application of convolutional neural networks with the Keras package.

APPENDIX A

Interactive Neural Networks

This appendix explores fundamental notions of neural networks with basic Python code—on the basis of both simple and shallow neural networks. The goal is to pro‐ vide a good grasp and intuition for important concepts that often disappear behind high-level, abstract APIs when working with standard machine and deep learning packages. The appendix has the following sections: • “Tensors and Tensor Operations” on page 407 covers the basics of tensors and the operations implemented on them. • “Simple Neural Networks” on page 409 discusses simple neural networks, or neu‐ ral networks that only have an input and an output layer. • “Shallow Neural Networks” on page 417 focuses on shallow neural networks, or neural networks with one hidden layer.

Tensors and Tensor Operations In addition to implementing several imports and configurations, the following Python code shows the four types of tensors relevant for the purposes of this appen‐ dix: scalar, vector, matrix, and cube tensors. Tensors are generally represented as potentially multidimensional ndarray objects in Python. For more details and exam‐ ples, see Chollet (2017, ch. 2): In [1]: import math import numpy as np import pandas as pd from pylab import plt, mpl np.random.seed(1)

407

plt.style.use('seaborn') mpl.rcParams['savefig.dpi'] = 300 mpl.rcParams['font.family'] = 'serif' np.set_printoptions(suppress=True) In [2]: t0 = np.array(10) t0 Out[2]: array(10) In [3]: t1 = np.array((2, 1)) t1 Out[3]: array([2, 1]) In [4]: t2 = np.arange(10).reshape(5, 2) t2 Out[4]: array([[0, 1], [2, 3], [4, 5], [6, 7], [8, 9]]) In [5]: t3 = np.arange(16).reshape(2, 4, 2) t3 Out[5]: array([[[ 0, 1], [ 2, 3], [ 4, 5], [ 6, 7]], [[ 8, [10, [12, [14,

9], 11], 13], 15]]])

Scalar tensor Vector tensor Matrix tensor Cube tensor In a neural network context, several mathematical operations on tensors are of importance, such as element-wise operations or the dot product: In [6]: t2 + 1 Out[6]: array([[ [ [ [ [

408

1, 2], 3, 4], 5, 6], 7, 8], 9, 10]])

| Appendix A: Interactive Neural Networks

In [7]: t2 + t2 Out[7]: array([[ 0, [ 4, [ 8, [12, [16,

2], 6], 10], 14], 18]])

In [8]: t1 Out[8]: array([2, 1]) In [9]: t2 Out[9]: array([[0, [2, [4, [6, [8,

1], 3], 5], 7], 9]])

In [10]: np.dot(t2, t1) Out[10]: array([ 1, 7, 13, 19, 25]) In [11]: t2[:, 0] * 2 + t2[:, 1] * 1 Out[11]: array([ 1, 7, 13, 19, 25]) In [12]: np.dot(t1, t2.T) Out[12]: array([ 1, 7, 13, 19, 25])

Broadcasting operation Element-wise operation Dot product with NumPy function Dot product in explicit notation

Simple Neural Networks Equipped with the basics of tensors, consider simple neural networks, which only have an input layer and an output layer.

Estimation The first problem is an estimation problem for which the labels are real-valued: In [13]: features = 3 In [14]: samples = 5 In [15]: l0 = np.random.random((samples, features)) l0 Out[15]: array([[0.417022 , 0.72032449, 0.00011437],

Interactive Neural Networks

|

409

[0.30233257, [0.18626021, [0.53881673, [0.20445225,

0.14675589, 0.34556073, 0.41919451, 0.87811744,

0.09233859], 0.39676747], 0.6852195 ], 0.02738759]])

In [16]: w = np.random.random((features, 1)) w Out[16]: array([[0.67046751], [0.4173048 ], [0.55868983]]) In [17]: l2 = np.dot(l0, w) l2 Out[17]: array([[0.58025848], [0.31553474], [0.49075552], [0.91901616], [0.51882238]]) In [18]: y = l0[:, 0] * 0.5 + l0[:, 1] y = y.reshape(-1, 1) y Out[18]: array([[0.9288355 ], [0.29792218], [0.43869083], [0.68860288], [0.98034356]])

Number of features Number of samples Random input layer Random weights Output layer via dot product Labels to be learned The following Python code goes step by step through a learning episode, from the calculation of the errors to the calculation of the mean-squared error (MSE) after the weights have been updated: In [19]: e = l2 - y e Out[19]: array([[-0.34857702], [ 0.01761256], [ 0.05206469], [ 0.23041328],

410

|

Appendix A: Interactive Neural Networks

[-0.46152118]]) In [20]: mse = (e ** 2).mean() mse Out[20]: 0.07812379019517127 In [21]: d = e * 1 d Out[21]: array([[-0.34857702], [ 0.01761256], [ 0.05206469], [ 0.23041328], [-0.46152118]]) In [22]: a = 0.01 In [23]: u = a * np.dot(l0.T, d) u Out[23]: array([[-0.0010055 ], [-0.00539194], [ 0.00167488]]) In [24]: w Out[24]: array([[0.67046751], [0.4173048 ], [0.55868983]]) In [25]: w -= u In [26]: w Out[26]: array([[0.67147301], [0.42269674], [0.55701495]]) In [27]: l2 = np.dot(l0, w) In [28]: e = l2 - y In [29]: mse = (e ** 2).mean() mse Out[29]: 0.07681782193617318

Errors in estimation MSE value given the estimation

Interactive Neural Networks

|

411

Backward propagation (here d = e)1 The learning rate The update values Weights before and after update New output layer (estimation) after update New error values after update New MSE values after update To improve the estimation, the same procedure needs to be repeated in general a larger number of times. In the following code, the learning rate is increased and the procedure is executed a few hundred times. The final MSE value is quite low and the estimation quite good: In [30]: a = 0.025 In [31]: w = np.random.random((features, 1)) w Out[31]: array([[0.14038694], [0.19810149], [0.80074457]]) In [32]: steps = 800 In [33]: for s in range(1, steps + 1): l2 = np.dot(l0, w) e = l2 - y u = a * np.dot(l0.T, e) w -= u mse = (e ** 2).mean() if s % 50 == 0: print(f'step={s:3d} | mse={mse:.5f}') step= 50 | mse=0.03064 step=100 | mse=0.01002 step=150 | mse=0.00390 step=200 | mse=0.00195 step=250 | mse=0.00124 step=300 | mse=0.00092 step=350 | mse=0.00074 step=400 | mse=0.00060

1 Since there is no hidden layer, backward propagation does take place with a factor of 1 as the value of the

derivative. Output and input layers are directly connected.

412

| Appendix A: Interactive Neural Networks

step=450 step=500 step=550 step=600 step=650 step=700 step=750 step=800

| | | | | | | |

mse=0.00050 mse=0.00041 mse=0.00035 mse=0.00029 mse=0.00024 mse=0.00020 mse=0.00017 mse=0.00014

In [34]: l2 - y Out[34]: array([[-0.01240168], [-0.01606065], [ 0.01274072], [-0.00087794], [ 0.01072845]]) In [35]: w Out[35]: array([[0.41907514], [1.02965827], [0.04421136]])

Adjusted learning rate Initial random weights Number of learning steps Residual errors of the estimation Final weights of the network

Classification The second problem is a classification problem for which the labels are binary and integer-valued. To improve the performance of the learning algorithm, a sigmoid function is used for activation (of the output layer). Figure A-1 shows the sigmoid function with its first derivative and compares it to a simple step function: In [36]: def sigmoid(x, deriv=False): if deriv: return sigmoid(x) * (1 - sigmoid(x)) return 1 / (1 + np.exp(-x)) In [37]: x = np.linspace(-10, 10, 100) In [38]: plt.figure(figsize=(10, 6)) plt.plot(x, np.where(x > 0, 1, 0), 'y--', label='step function') plt.plot(x, sigmoid(x), 'r', label='sigmoid') plt.plot(x, sigmoid(x, True), '--', label='derivative') plt.legend();

Interactive Neural Networks

|

413

Figure A-1. Step function, sigmoid function, and its first derivative To keep things simple, the classification problem is based on random binary features and binary labels data. Apart from the different features and labels data, only the acti‐ vation for the output layer is different from the estimation problem. The learning algorithm for the updating of the neural networks’ weights is basically the same: In [39]: features = 4 samples = 5 In [40]: l0 = np.random.randint(0, 2, (samples, features)) l0 Out[40]: array([[1, 1, 1, 1], [0, 1, 1, 0], [0, 1, 0, 0], [1, 1, 1, 0], [1, 0, 0, 1]]) In [41]: w = np.random.random((features, 1)) w Out[41]: array([[0.42110763], [0.95788953], [0.53316528], [0.69187711]]) In [42]: l2 = sigmoid(np.dot(l0, w)) l2 Out[42]: array([[0.93112111], [0.81623654], [0.72269905], [0.87126189], [0.75268514]])

414

|

Appendix A: Interactive Neural Networks

In [43]: l2.round() Out[43]: array([[1.], [1.], [1.], [1.], [1.]]) In [44]: y = np.random.randint(0, 2, samples) y = y.reshape(-1, 1) y Out[44]: array([[1], [1], [0], [0], [0]]) In [45]: e = l2 - y e Out[45]: array([[-0.06887889], [-0.18376346], [ 0.72269905], [ 0.87126189], [ 0.75268514]]) In [46]: mse = (e ** 2).mean() mse Out[46]: 0.37728788783411127 In [47]: a = 0.02 In [48]: d = e * sigmoid(l2, True) d Out[48]: array([[-0.01396723], [-0.03906484], [ 0.15899479], [ 0.18119776], [ 0.16384833]]) In [49]: u = a * np.dot(l0.T, d) u Out[49]: array([[0.00662158], [0.00574321], [0.00256331], [0.00299762]]) In [50]: w Out[50]: array([[0.42110763], [0.95788953], [0.53316528], [0.69187711]])

Interactive Neural Networks

|

415

In [51]: w -= u In [52]: w Out[52]: array([[0.41448605], [0.95214632], [0.53060197], [0.68887949]])

Input layer with binary features Sigmoid activated output layer Binary labels data Backward propagation via the first derivative As before, a loop with a larger number of iterations for the learning step is required to get to accurate classification results. Depending on the random numbers drawn, an accuracy of 100% is possible, as in the following example: In [53]: steps = 3001 In [54]: a = 0.025 In [55]: w = np.random.random((features, 1)) w Out[55]: array([[0.41253884], [0.03417131], [0.62402999], [0.66063573]]) In [56]: for s in range(1, steps + 1): l2 = sigmoid(np.dot(l0, w)) e = l2 - y d = e * sigmoid(l2, True) u = a * np.dot(l0.T, d) w -= u mse = (e ** 2).mean() if s % 200 == 0: print(f'step={s:4d} | mse={mse:.4f}') step= 200 | mse=0.1899 step= 400 | mse=0.1572 step= 600 | mse=0.1349 step= 800 | mse=0.1173 step=1000 | mse=0.1029 step=1200 | mse=0.0908 step=1400 | mse=0.0806 step=1600 | mse=0.0720 step=1800 | mse=0.0646 step=2000 | mse=0.0583 step=2200 | mse=0.0529

416

|

Appendix A: Interactive Neural Networks

step=2400 step=2600 step=2800 step=3000

| | | |

mse=0.0482 mse=0.0441 mse=0.0405 mse=0.0373

In [57]: l2 Out[57]: array([[0.71220474], [0.92308745], [0.16614971], [0.20193503], [0.17094583]]) In [58]: l2.round() == y Out[58]: array([[ True], [ True], [ True], [ True], [ True]]) In [59]: w Out[59]: array([[-3.86002022], [-1.61346536], [ 4.09895004], [ 2.28088807]])

Shallow Neural Networks The neural network of the previous section consists only of an input layer and an out‐ put layer. In other words, input and output layers are directly connected. A shallow neural network has one hidden layer that is between the input and output layer. Given this structure, two sets of weights are required to connect the total of three layers in the neural network. This section analyzes shallow neural networks for estimation and classification.

Estimation As in the previous section, let’s take the estimation problem first. The following Python code builds the neural network with the three layers and the two sets of weights. This first sequence of steps is usually called forward propagation. The input layer matrix in this context is in general of full rank, indicating that a perfect estima‐ tion result is possible: In [60]: features = 5 samples = 5 In [61]: l0 = np.random.random((samples, features)) l0 Out[61]: array([[0.29849529, 0.44613451, 0.22212455, 0.07336417, 0.46923853], [0.09617226, 0.90337017, 0.11949047, 0.52479938, 0.083623 ],

Interactive Neural Networks

|

417

[0.91686133, 0.91044838, 0.29893011, 0.58438912, 0.56591203], [0.61393832, 0.95653566, 0.26097898, 0.23101542, 0.53344849], [0.94993814, 0.49305959, 0.54060051, 0.7654851 , 0.04534573]]) In [62]: np.linalg.matrix_rank(l0) Out[62]: 5 In [63]: units = 3 In [64]: w0 = np.random.random((features, units)) w0 Out[64]: array([[0.13996612, 0.79240359, 0.02980136], [0.88312548, 0.54078819, 0.44798018], [0.89213587, 0.37758434, 0.53842469], [0.65229888, 0.36126102, 0.57100856], [0.63783648, 0.12631489, 0.69020459]]) In [65]: l1 = np.dot(l0, w0) l1 Out[65]: array([[0.98109007, [1.31351565, [1.94121167, [1.65444429, [1.57892999,

0.64743919, 0.81000928, 1.61435539, 1.25315104, 1.50576525,

0.69411448], 0.82927653], 1.32042417], 1.08742312], 1.00865941]])

In [66]: w1 = np.random.random((units, 1)) w1 Out[66]: array([[0.6477494 ], [0.35393909], [0.76323305]]) In [67]: l2 = np.dot(l1, w1) l2 Out[67]: array([[1.39442565], [1.77045418], [2.83659354], [2.3451617 ], [2.32554234]]) In [68]: y = np.random.random((samples, 1)) y Out[68]: array([[0.35653172], [0.75278835], [0.88134183], [0.01166919], [0.49810907]])

The random input layer The rank of the input layer matrix

418

|

Appendix A: Interactive Neural Networks

The number of hidden units The first set of random weights, given the features and units parameters The hidden layer, given the input layer and the weights The second set of random weights The output layer, given the hidden layer and the weights The random labels data The second sequence of steps is usually called backward propagation—with respect to the estimation errors. The two sets of weights are updated, starting at the output layer and updating the set of weights w1 between the hidden layer and the output layer. Afterwards, taking the updated weights w1 into account, the set of weights w0 between the input layer and the hidden layer is updated: In [69]: e2 = l2 - y e2 Out[69]: array([[1.03789393], [1.01766583], [1.95525171], [2.33349251], [1.82743327]]) In [70]: mse = (e2 ** 2).mean() mse Out[70]: 2.9441152813655007 In [71]: d2 = e2 * 1 d2 Out[71]: array([[1.03789393], [1.01766583], [1.95525171], [2.33349251], [1.82743327]]) In [72]: a = 0.05 In [73]: u2 = a * np.dot(l1.T, d2) u2 Out[73]: array([[0.64482837], [0.51643336], [0.42634283]]) In [74]: w1 Out[74]: array([[0.6477494 ], [0.35393909],

Interactive Neural Networks

|

419

[0.76323305]]) In [75]: w1 -= u2 In [76]: w1 Out[76]: array([[ 0.00292103], [-0.16249427], [ 0.33689022]]) In [77]: e1 = np.dot(d2, w1.T) In [78]: d1 = e1 * 1 In [79]: u1 = a * np.dot(l0.T, d1) In [80]: w0 -= u1 In [81]: w0 Out[81]: array([[ [ [ [ [

0.13918198, 0.88220599, 0.89176585, 0.65175984, 0.63739741,

0.8360247 , -0.06063583], 0.59193836, 0.34193342], 0.39816855, 0.49574861], 0.39124762, 0.50883904], 0.15074009, 0.63956519]])

Update procedure for the set of weights w1 Update procedure for the set of weights w0 The following Python code implements the learning (that is, the updating of the net‐ work weights) as a for loop with a larger number of iterations. By increasing the number of iterations, the estimation results can be made arbitrarily precise: In [82]: a = 0.015 steps = 5000 In [83]: for s in range(1, steps + 1): l1 = np.dot(l0, w0) l2 = np.dot(l1, w1) e2 = l2 - y u2 = a * np.dot(l1.T, e2) w1 -= u2 e1 = np.dot(e2, w1.T) u1 = a * np.dot(l0.T, e1) w0 -= u1 mse = (e2 ** 2).mean() if s % 750 == 0: print(f'step={s:5d} | mse={mse:.6f}') step= 750 | mse=0.039263 step= 1500 | mse=0.009867 step= 2250 | mse=0.000666 step= 3000 | mse=0.000027 step= 3750 | mse=0.000001

420

|

Appendix A: Interactive Neural Networks

step= 4500 | mse=0.000000 In [84]: l2 Out[84]: array([[0.35634333], [0.75275415], [0.88135507], [0.01179945], [0.49809208]]) In [85]: y Out[85]: array([[0.35653172], [0.75278835], [0.88134183], [0.01166919], [0.49810907]]) In [86]: (l2 - y) Out[86]: array([[-0.00018839], [-0.00003421], [ 0.00001324], [ 0.00013025], [-0.00001699]])

Classification Next is the classification problem. The implementation in this context is pretty close to the estimation problem. However, the sigmoid function is used again for activa‐ tion. The following Python code generates the random sample data first: In [87]: features = 5 samples = 10 units = 10 In [88]: np.random.seed(200) l0 = np.random.randint(0, 2, (samples, features)) w0 = np.random.random((features, units)) w1 = np.random.random((units, 1)) y = np.random.randint(0, 2, (samples, 1)) In [89]: l0 Out[89]: array([[0, [1, [1, [0, [1, [1, [0, [0, [0, [0,

1, 0, 1, 0, 1, 1, 1, 1, 1, 0,

0, 1, 1, 1, 1, 0, 0, 0, 1, 1,

0, 1, 1, 1, 1, 1, 1, 0, 1, 0,

0], 0], 0], 1], 0], 0], 0], 1], 1], 0]])

In [90]: y

Interactive Neural Networks

|

421

Out[90]: array([[1], [0], [1], [0], [1], [0], [0], [0], [1], [1]])

Binary features data (input layer) Binary labels data The implementation of the learning algorithm again makes use of a for loop to repeat the weights-updating step as often as necessary. Depending on the random numbers generated for the features and labels data, an accuracy of 100% can be achieved after enough learning steps: In [91]: a = 0.1 steps = 20000 In [92]: for s in range(1, steps + 1): l1 = sigmoid(np.dot(l0, w0)) l2 = sigmoid(np.dot(l1, w1)) e2 = l2 - y d2 = e2 * sigmoid(l2, True) u2 = a * np.dot(l1.T, d2) w1 -= u2 e1 = np.dot(d2, w1.T) d1 = e1 * sigmoid(l1, True) u1 = a * np.dot(l0.T, d1) w0 -= u1 mse = (e2 ** 2).mean() if s % 2000 == 0: print(f'step={s:5d} | mse={mse:.5f}') step= 2000 | mse=0.00933 step= 4000 | mse=0.02399 step= 6000 | mse=0.05134 step= 8000 | mse=0.00064 step=10000 | mse=0.00013 step=12000 | mse=0.00009 step=14000 | mse=0.00007 step=16000 | mse=0.00007 step=18000 | mse=0.00012 step=20000 | mse=0.00015 In [93]: acc = l2.round() == y acc Out[93]: array([[ True],

422

|

Appendix A: Interactive Neural Networks

[ [ [ [ [ [ [ [ [

True], True], True], True], True], True], True], True], True]])

In [94]: sum(acc) / len(acc) Out[94]: array([1.])

Forward propagation Backward propagation Accuracy of the classification

References Books cited in this appendix: Chollet, Francois. 2017. Deep Learning with Python. Shelter Island: Manning.

Interactive Neural Networks

|

423

APPENDIX B

Neural Network Classes

Building on the foundations from Appendix A, this appendix provides simple, classbased implementations of neural networks that mimic the APIs of packages such as scikit-learn. The implementation is based on pure, simple Python code and is for illustration and instruction. The classes presented in this appendix cannot replace robust, efficient, and scalable implementations found in the standard Python pack‐ ages, such as scikit-learn or TensorFlow in combination with Keras. The appendix comprises the following sections: • “Activation Functions” on page 425 introduces a Python function with different activation functions. • “Simple Neural Networks” on page 426 presents a Python class for simple neural networks. • “Shallow Neural Networks” on page 431 presents a Python class for shallow neural networks. • “Predicting Market Direction” on page 435 applies the class for shallow neural networks to financial data. The implementations and examples in this appendix are simple and straightforward. The Python classes are not well suited to attack larger estimation or classification problems. The idea is rather to show easy-to-understand Python implementations from scratch.

Activation Functions Appendix A uses two activation functions implicitly or explicitly: linear function and sigmoid function. The Python function activation adds the relu (rectified linear 425

unit) and softplus functions to the set of options. For all these activation functions, the first derivative is also defined: In [1]: import math import numpy as np import pandas as pd from pylab import plt, mpl plt.style.use('seaborn') mpl.rcParams['savefig.dpi'] = 300 mpl.rcParams['font.family'] = 'serif' np.set_printoptions(suppress=True) In [2]: def activation(x, act='linear', deriv=False): if act == 'sigmoid': if deriv: out = activation(x, 'sigmoid', False) return out * (1 - out) return 1 / (1 + np.exp(-x)) elif act == 'relu': if deriv: return np.where(x > 0, 1, 0) return np.maximum(x, 0) elif act == 'softplus': if deriv: return activation(x, act='sigmoid') return np.log(1 + np.exp(x)) elif act == 'linear': if deriv: return 1 return x else: raise ValueError('Activation function not known.') In [3]: x = np.linspace(-1, 1, 20) In [4]: activation(x, 'sigmoid') Out[4]: array([0.26894142, 0.29013328, 0.38374461, 0.40892261, 0.51315486, 0.53939188, 0.64082516, 0.66467779,

0.31228169, 0.43458759, 0.56541241, 0.68771831,

0.33532221, 0.46060812, 0.59107739, 0.70986672,

0.35917484, 0.48684514, 0.61625539, 0.73105858])

In [5]: activation(x, 'sigmoid', True) Out[5]: array([0.19661193, 0.20595596, 0.23648468, 0.24170491, 0.24982695, 0.24844828, 0.23016827, 0.22288122,

0.21476184, 0.24572122, 0.24572122, 0.21476184,

0.22288122, 0.24844828, 0.24170491, 0.20595596,

0.23016827, 0.24982695, 0.23648468, 0.19661193])

Simple Neural Networks This section presents a class for simple neural networks that has an API similar to those of models from standard Python packages for machine or deep learning (in 426

|

Appendix B: Neural Network Classes

particular, scikit-learn and Keras). Consider the class sinn as presented in the fol‐ lowing Python code. It implements a simple neural network and defines the two main methods .fit() and .predict(). The .metrics() method calculates typical perfor‐ mance metrics: the mean-squared error (MSE) for estimation and the accuracy for classification. The class also implements two methods for the forward and backward propagation steps: In [6]: class sinn: def __init__(self, act='linear', lr=0.01, steps=100, verbose=False, psteps=200): self.act = act self.lr = lr self.steps = steps self.verbose = verbose self.psteps = psteps def forward(self): ''' Forward propagation. ''' self.l2 = activation(np.dot(self.l0, self.w), self.act) def backward(self): ''' Backward propagation. ''' self.e = self.l2 - self.y d = self.e * activation(self.l2, self.act, True) u = self.lr * np.dot(self.l0.T, d) self.w -= u def metrics(self, s): ''' Performance metrics. ''' mse = (self.e ** 2).mean() acc = float(sum(self.l2.round() == self.y) / len(self.y)) self.res = self.res.append( pd.DataFrame({'mse': mse, 'acc': acc}, index=[s,]) ) if s % self.psteps == 0 and self.verbose: print(f'step={s:5d} | mse={mse:.6f}') print(f' | acc={acc:.6f}') def fit(self, l0, y, steps=None, seed=None): ''' Fitting step. ''' self.l0 = l0 self.y = y if steps is None: steps = self.steps self.res = pd.DataFrame() samples, features = l0.shape if seed is not None: np.random.seed(seed) self.w = np.random.random((features, 1)) for s in range(1, steps + 1): self.forward()

Neural Network Classes

|

427

self.backward() self.metrics(s) def predict(self, X): ''' Prediction step. ''' return activation(np.dot(X, self.w), self.act)

Estimation First is an estimation problem that can be solved by the use of regression techniques: In [7]: features = 5 samples = 5 In [8]: np.random.seed(10) l0 = np.random.standard_normal((samples, features)) l0 Out[8]: array([[ 1.3315865 , 0.71527897, -1.54540029, -0.00838385, 0.62133597], [-0.72008556, 0.26551159, 0.10854853, 0.00429143, -0.17460021], [ 0.43302619, 1.20303737, -0.96506567, 1.02827408, 0.22863013], [ 0.44513761, -1.13660221, 0.13513688, 1.484537 , -1.07980489], [-1.97772828, -1.7433723 , 0.26607016, 2.38496733, 1.12369125]]) In [9]: np.linalg.matrix_rank(l0) Out[9]: 5 In [10]: y = np.random.random((samples, 1)) y Out[10]: array([[0.8052232 ], [0.52164715], [0.90864888], [0.31923609], [0.09045935]]) In [11]: reg = np.linalg.lstsq(l0, y, rcond=-1)[0] In [12]: reg Out[12]: array([[-0.74919308], [ 0.00146473], [-1.49864704], [-0.02498757], [-0.82793882]]) In [13]: np.allclose(np.dot(l0, reg), y) Out[13]: True

Exact solution by regression

428

|

Appendix B: Neural Network Classes

Applying the sinn class to the estimation problem requires quite some effort in the form of repeated learning steps. However, by increasing the number of steps, one can make the estimate arbitrarily precise: In [14]: model = sinn(lr=0.015, act='linear', steps=6000, verbose=True, psteps=1000) In [15]: %time model.fit(l0, y, seed=100) step= 1000 | mse=0.008086 | acc=0.000000 step= 2000 | mse=0.000545 | acc=0.000000 step= 3000 | mse=0.000037 | acc=0.000000 step= 4000 | mse=0.000002 | acc=0.000000 step= 5000 | mse=0.000000 | acc=0.000000 step= 6000 | mse=0.000000 | acc=0.000000 CPU times: user 5.23 s, sys: 29.7 ms, total: 5.26 s Wall time: 5.26 s In [16]: model.predict(l0) Out[16]: array([[0.80512489], [0.52144986], [0.90872498], [0.31919803], [0.09045743]]) In [17]: model.predict(l0) - y Out[17]: array([[-0.0000983 ], [-0.00019729], [ 0.0000761 ], [-0.00003806], [-0.00000191]])

Residual errors of the neural network estimation

Classification Second is a classification problem that can also be attacked with the sinn class. Here, standard regression techniques are in general of no use. For the particular set of ran‐ dom features and labels, the sinn model reaches an accuracy of 100%. Again, quite some effort is required in the form of repeated learning steps. Figure B-1 shows how the prediction accuracy changes with the number of learning steps: In [18]: features = 5 samples = 10 In [19]: np.random.seed(3)

Neural Network Classes

|

429

l0 = np.random.randint(0, 2, (samples, features)) l0 Out[19]: array([[0, 0, 1, 1, 0], [0, 0, 1, 1, 1], [0, 1, 1, 1, 0], [1, 1, 0, 0, 0], [0, 1, 1, 0, 0], [0, 1, 0, 0, 0], [0, 1, 0, 1, 1], [0, 1, 0, 0, 1], [1, 0, 0, 1, 0], [1, 0, 1, 1, 1]]) In [20]: np.linalg.matrix_rank(l0) Out[20]: 5 In [21]: y = np.random.randint(0, 2, (samples, 1)) y Out[21]: array([[1], [0], [1], [0], [0], [1], [1], [1], [0], [0]]) In [22]: model = sinn(lr=0.01, act='sigmoid') In [23]: %time model.fit(l0, y, 4000) CPU times: user 3.57 s, sys: 9.6 ms, total: 3.58 s Wall time: 3.59 s In [24]: model.l2 Out[24]: array([[0.51118415], [0.34390898], [0.84733758], [0.07601979], [0.40505454], [0.84145926], [0.95592461], [0.72680243], [0.11219587], [0.00806003]]) In [25]: model.predict(l0).round() == y Out[25]: array([[ True], [ True], [ True], [ True],

430

|

Appendix B: Neural Network Classes

[ [ [ [ [ [

True], True], True], True], True], True]])

In [26]: ax = model.res['acc'].plot(figsize=(10, 6), title='Prediction Accuracy | Classification') ax.set(xlabel='steps', ylabel='accuracy');

The sigmoid function is used for activation Perfect accuracy on this particular data set

Figure B-1. Prediction accuracy versus the number of learning steps

Shallow Neural Networks This section applies the class shnn, which implements shallow neural networks with one hidden layer, to estimation and classification problems. The class structure is along the lines of the sinn class from the previous section: In [27]: class shnn: def __init__(self, units=12, act='linear', lr=0.01, steps=100, verbose=False, psteps=200, seed=None): self.units = units self.act = act self.lr = lr self.steps = steps self.verbose = verbose

Neural Network Classes

|

431

def

def

def

def

def

def

432

|

self.psteps = psteps self.seed = seed initialize(self): ''' Initializes the random weights. ''' if self.seed is not None: np.random.seed(self.seed) samples, features = self.l0.shape self.w0 = np.random.random((features, self.units)) self.w1 = np.random.random((self.units, 1)) forward(self): ''' Forward propagation. ''' self.l1 = activation(np.dot(self.l0, self.w0), self.act) self.l2 = activation(np.dot(self.l1, self.w1), self.act) backward(self): ''' Backward propagation. ''' self.e = self.l2 - self.y d2 = self.e * activation(self.l2, self.act, True) u2 = self.lr * np.dot(self.l1.T, d2) self.w1 -= u2 e1 = np.dot(d2, self.w1.T) d1 = e1 * activation(self.l1, self.act, True) u1 = self.lr * np.dot(self.l0.T, d1) self.w0 -= u1 metrics(self, s): ''' Performance metrics. ''' mse = (self.e ** 2).mean() acc = float(sum(self.l2.round() == self.y) / len(self.y)) self.res = self.res.append( pd.DataFrame({'mse': mse, 'acc': acc}, index=[s,]) ) if s % self.psteps == 0 and self.verbose: print(f'step={s:5d} | mse={mse:.5f}') print(f' | acc={acc:.5f}') fit(self, l0, y, steps=None): ''' Fitting step. ''' self.l0 = l0 self.y = y if steps is None: steps = self.steps self.res = pd.DataFrame() self.initialize() self.forward() for s in range(1, steps + 1): self.backward() self.forward() self.metrics(s) predict(self, X):

Appendix B: Neural Network Classes

''' Prediction step. ''' l1 = activation(np.dot(X, self.w0), self.act) l2 = activation(np.dot(l1, self.w1), self.act) return l2

Estimation Again, the estimation problem comes first. For 5 features and 10 samples, a perfect regression solution is unlikely to exist. As a result, the MSE value of the regression is relatively high: In [28]: features = 5 samples = 10 In [29]: l0 = np.random.standard_normal((samples, features)) In [30]: np.linalg.matrix_rank(l0) Out[30]: 5 In [31]: y = np.random.random((samples, 1)) In [32]: reg = np.linalg.lstsq(l0, y, rcond=-1)[0] In [33]: (np.dot(l0, reg) - y) Out[33]: array([[-0.10226341], [-0.42357164], [-0.25150491], [-0.30984143], [-0.85213261], [-0.13791373], [-0.52336502], [-0.50304204], [-0.7728686 ], [-0.3716898 ]]) In [34]: ((np.dot(l0, reg) - y) ** 2).mean() Out[34]: 0.23567187607888118

However, the shallow neural network estimate based on the shnn class is quite good and shows a relatively low MSE value compared to the regression value: In [35]: model = shnn(lr=0.01, units=16, act='softplus', verbose=True, psteps=2000, seed=100) In [36]: %time model.fit(l0, y, 8000) step= 2000 | mse=0.00205 | acc=0.00000 step= 4000 | mse=0.00098 | acc=0.00000 step= 6000 | mse=0.00043 | acc=0.00000

Neural Network Classes

|

433

step= 8000 | mse=0.00022 | acc=0.00000 CPU times: user 8.15 s, sys: 69.2 ms, total: 8.22 s Wall time: 8.3 s In [37]: model.l2 - y Out[37]: array([[-0.00390976], [-0.00522077], [ 0.02053932], [-0.0042113 ], [-0.0006624 ], [-0.01001395], [ 0.01783203], [-0.01498316], [-0.0177866 ], [ 0.02782519]])

Classification The classification example takes the estimation numbers and applies rounding to them. The shallow neural network converges quickly to predict the labels with 100% accuracy (see Figure B-2): In [38]: model = shnn(lr=0.025, act='sigmoid', steps=200, verbose=True, psteps=50, seed=100) In [39]: l0.round() Out[39]: array([[ 0., [-1., [ 0., [-0., [ 1., [ 1., [-1., [ 1., [-1., [ 0.,

-1., -2., 1., 0., -1., -1., -0., 2., 0., 0.,

-2., -0., -1., -1., 1., 1., 1., -1., 0., -0.,

1., -0., -1., -0., 1., -2., -1., -0., 0., 1.,

In [40]: np.linalg.matrix_rank(l0) Out[40]: 5 In [41]: y.round() Out[41]: array([[0.], [1.], [1.], [1.], [1.], [1.], [0.], [1.], [0.], [0.]])

434

|

Appendix B: Neural Network Classes

-0.], -2.], -1.], -1.], -1.], 1.], 1.], -0.], 2.], 1.]])

In [42]: model.fit(l0.round(), y.round()) step= 50 | mse=0.26774 | acc=0.60000 step= 100 | mse=0.22556 | acc=0.60000 step= 150 | mse=0.19939 | acc=0.70000 step= 200 | mse=0.16924 | acc=1.00000 In [43]: ax = model.res.plot(figsize=(10, 6), secondary_y='mse') ax.get_legend().set_bbox_to_anchor((0.2, 0.5));

Figure B-2. Performance metrics for the shallow neural network (classification)

Predicting Market Direction This section applies the shnn class to predict the future direction of the EUR/USD exchange rate. The analysis is in-sample only to illustrate the application of shnn to real-world data. See Chapter 10 for the implementation of a more realistic setup for the vectorized backtesting of such prediction-based strategies. The following Python code imports the financial data—10 years’ worth of EOD data —and creates lagged, normalized log returns used as the features. The labels data is the direction of the price series as a binary data set: In [44]: url = 'http://hilpisch.com/aiif_eikon_eod_data.csv' In [45]: raw = pd.read_csv(url, index_col=0, parse_dates=True).dropna() In [46]: sym = 'EUR='

Neural Network Classes

|

435

In [47]: data = pd.DataFrame(raw[sym]) In [48]: lags = 5 cols = [] data['r'] = np.log(data / data.shift(1)) data['d'] = np.where(data['r'] > 0, 1, 0) for lag in range(1, lags + 1): col = f'lag_{lag}' data[col] = data['r'].shift(lag) cols.append(col) data.dropna(inplace=True) data[cols] = (data[cols] - data[cols].mean()) / data[cols].std() In [49]: data.head() Out[49]: Date 2010-01-12 2010-01-13 2010-01-14 2010-01-15 2010-01-19

EUR= 1.4494 1.4510 1.4502 1.4382 1.4298

r

d

lag_1

lag_2

lag_3

lag_4

\

-0.001310 0 1.256582 1.177935 -1.142025 0.560551 0.001103 1 -0.214533 1.255944 1.178974 -1.142118 -0.000551 0 0.213539 -0.214803 1.256989 1.178748 -0.008309 0 -0.079986 0.213163 -0.213853 1.256758 -0.005858 0 -1.456028 -0.080289 0.214140 -0.214000

lag_5 Date 2010-01-12 2010-01-13 2010-01-14 2010-01-15 2010-01-19

-0.511372 0.560740 -1.141841 1.178904 1.256910

Market direction as the labels data Lagged log returns as the features data Gaussian normalization of the features data With the data preprocessing accomplished, the application of the shallow neural net‐ work class shnn for a supervised classification is straightforward. Figure B-3 shows that the prediction-based strategy in-sample significantly outperforms the passive benchmark investment: In [50]: model = shnn(lr=0.0001, act='sigmoid', steps=10000, verbose=True, psteps=2000, seed=100) In [51]: y = data['d'].values.reshape(-1, 1) In [52]: %time model.fit(data[cols].values, y) step= 2000 | mse=0.24964 | acc=0.51594 step= 4000 | mse=0.24951 | acc=0.52390

436

| Appendix B: Neural Network Classes

step= 6000 | mse=0.24945 | acc=0.52231 step= 8000 | mse=0.24940 | acc=0.52510 step=10000 | mse=0.24936 | acc=0.52430 CPU times: user 9min 1s, sys: 40.9 s, total: 9min 42s Wall time: 1min 21s In [53]: data['p'] = np.where(model.predict(data[cols]) > 0.5, 1, -1) In [54]: data['p'].value_counts() Out[54]: 1 1257 -1 1253 Name: p, dtype: int64 In [55]: data['s'] = data['p'] * data['r'] In [56]: data[['r', 's']].sum().apply(np.exp) Out[56]: r 0.772411 s 1.885677 dtype: float64 In [57]: data[['r', 's']].cumsum().apply(np.exp).plot(figsize=(10, 6));

Derives the position values from the prediction values Calculates the strategy returns from the position values and the log returns Calculates the gross performance of the strategy and the benchmark investment Shows the gross performance of the strategy and the benchmark investment over time

Neural Network Classes

|

437

Figure B-3. Gross performance of prediction-based strategy compared to passive bench‐ mark investment (in-sample)

438

|

Appendix B: Neural Network Classes

APPENDIX C

Convolutional Neural Networks

Part III focuses on dense neural networks (DNNs) and recurrent neural networks (RNNs) as two standard types of neural networks. The charm of DNNs lies in the fact that they are good universal approximators. The examples in the book for reinforce‐ ment learning, for instance, make use of DNNs to approximate the optimal action policy. On the other hand, RNNs are specifically designed to handle sequential data, such as time series data. This is helpful when trying, for example, to predict future values of financial time series. However, convolutional neural networks (CNNs) are another standard type of neural network that is widely used in practice. They have been particularly successful, among other domains, in computer vision. CNNs were able to set new benchmarks in a number of standard tests and challenges, such as the ImageNet Challenge; for more on this, see The Economist (2016) or Gerrish (2018). Computer vision in turn is important in such domains as autonomous vehicles or security and surveillance. This brief appendix illustrates the application of a CNN to the prediction of financial time series data. For details on CNNs, see Chollet (2017, ch. 5) and Goodfellow et al. (2016, ch. 9).

Features and Labels Data The following Python code first takes care of the required imports and customiza‐ tions. It then imports the data set that contains end-of-day (EOD) data for a number of financial instruments. This data set is used throughout the book for different examples: In [1]: import import import import

os math numpy as np pandas as pd

439

from pylab import plt, mpl plt.style.use('seaborn') mpl.rcParams['savefig.dpi'] = 300 mpl.rcParams['font.family'] = 'serif' os.environ['PYTHONHASHSEED'] = '0' In [2]: url = 'http://hilpisch.com/aiif_eikon_eod_data.csv' In [3]: symbol = 'EUR=' In [4]: data = pd.DataFrame(pd.read_csv(url, index_col=0, parse_dates=True).dropna()[symbol]) In [5]: data.info()

DatetimeIndex: 2516 entries, 2010-01-04 to 2019-12-31 Data columns (total 1 columns): # Column Non-Null Count Dtype --- ------ -------------- ----0 EUR= 2516 non-null float64 dtypes: float64(1) memory usage: 39.3 KB

Retrieves and selects the financial time series data The next step is to generate the features data, lag the data, split it into training and test data sets, and finally normalize it based on the statistics of the training data set: In [6]: lags = 5 In [7]: features = [symbol, 'r', 'd', 'sma', 'min', 'max', 'mom', 'vol'] In [8]: def add_lags(data, symbol, lags, window=20, features=features): cols = [] df = data.copy() df.dropna(inplace=True) df['r'] = np.log(df / df.shift(1)) df['sma'] = df[symbol].rolling(window).mean() df['min'] = df[symbol].rolling(window).min() df['max'] = df[symbol].rolling(window).max() df['mom'] = df['r'].rolling(window).mean() df['vol'] = df['r'].rolling(window).std() df.dropna(inplace=True) df['d'] = np.where(df['r'] > 0, 1, 0) for f in features: for lag in range(1, lags + 1): col = f'{f}_lag_{lag}' df[col] = df[f].shift(lag) cols.append(col) df.dropna(inplace=True) return df, cols

440

|

Appendix C: Convolutional Neural Networks

In [9]: data, cols = add_lags(data, symbol, lags, window=20, features=features) In [10]: split = int(len(data) * 0.8) In [11]: train = data.iloc[:split].copy() In [12]: mu, std = train[cols].mean(), train[cols].std() In [13]: train[cols] = (train[cols] - mu) / std In [14]: test = data.iloc[split:].copy() In [15]: test[cols] = (test[cols] - mu) / std

Simple moving average feature Rolling minimum value feature Rolling maximum value feature Time series momentum feature Rolling volatility feature Gaussian normalization of training data set Gaussian normalization of test data set

Training the Model The implementation of CNNs is similar to that of DNNs. First, the Python code that follows takes care of the imports from Keras and the definition of the function to set all relevant seed values of the random number generators: In [16]: import random import tensorflow as tf from keras.models import Sequential from keras.layers import Dense, Conv1D, Flatten Using TensorFlow backend. In [17]: def set_seeds(seed=100): random.seed(seed) np.random.seed(seed) tf.random.set_seed(seed)

Convolutional Neural Networks

|

441

The following Python code implements and trains a simple CNN. At the core of the model is a one-dimensional convolutional layer that is suited for time series data (see Keras convolutional layers for details): In [18]: set_seeds() model = Sequential() model.add(Conv1D(filters=96, kernel_size=5, activation='relu', input_shape=(len(cols), 1))) model.add(Flatten()) model.add(Dense(10, activation='relu')) model.add(Dense(1, activation='sigmoid')) model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy']) In [19]: model.summary() Model: "sequential_1" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv1d_1 (Conv1D) (None, 36, 96) 576 _________________________________________________________________ flatten_1 (Flatten) (None, 3456) 0 _________________________________________________________________ dense_1 (Dense) (None, 10) 34570 _________________________________________________________________ dense_2 (Dense) (None, 1) 11 ================================================================= Total params: 35,157 Trainable params: 35,157 Non-trainable params: 0 _________________________________________________________________ In [20]: %%time model.fit(np.atleast_3d(train[cols]), train['d'], epochs=60, batch_size=48, verbose=False, validation_split=0.15, shuffle=False) CPU times: user 10.1 s, sys: 1.87 s, total: 12 s Wall time: 4.78 s Out[20]:

Figure C-1 presents the performance metrics for the training and validation data sets over the different training epochs: In [21]: res = pd.DataFrame(model.history.history) In [22]: res.tail(3) Out[22]: val_loss val_accuracy loss accuracy 57 0.699932 0.508361 0.635633 0.597165 58 0.719671 0.501672 0.634539 0.598937

442

|

Appendix C: Convolutional Neural Networks

59

0.729954

0.505017

0.634403

0.601890

In [23]: res.plot(figsize=(10, 6));

Figure C-1. Performance metrics for the training and validation of the CNN

Testing the Model Finally, the Python code that follows applies the trained model to the test data set. The CNN model outperforms the passive benchmark investment significantly. How‐ ever, taking into account transaction costs in the form of typical (retail) bid-ask spreads, it eats up larger parts of the outperformance. Figure C-2 visualizes the per‐ formances over time: In [24]: model.evaluate(np.atleast_3d(test[cols]), test['d']) 499/499 [==============================] - 0s 25us/step Out[24]: [0.7364848222665653, 0.5210421085357666] In [25]: test['p'] = np.where(model.predict(np.atleast_3d(test[cols])) > 0.5, 1, 0) In [26]: test['p'] = np.where(test['p'] > 0, 1, -1) In [27]: test['p'].value_counts() Out[27]: -1 478 1 21 Name: p, dtype: int64 In [28]: (test['p'].diff() != 0).sum() Out[28]: 41 In [29]: test['s'] = test['p'] * test['r']

Convolutional Neural Networks

|

443

In [30]: ptc = 0.00012 / test[symbol] In [31]: test['s_'] = np.where(test['p'] != 0, test['s'] - ptc, test['s']) In [32]: test[['r', 's', 's_']].sum().apply(np.exp) Out[32]: r 0.931992 s 1.086525 s_ 1.031307 dtype: float64 In [33]: test[['r', 's', 's_']].cumsum().apply(np.exp).plot(figsize=(10, 6));

The accuracy ratio out-of-sample The positions (long/short) based on the predictions The number of trades resulting from the positions Proportional transaction costs for given bid-ask spread The strategy performance before transaction costs The strategy performance after transaction costs

Figure C-2. Gross performance of passive benchmark investment and CNN strategy (before/after transaction costs)

444

|

Appendix C: Convolutional Neural Networks

Resources Books and papers cited in this appendix: Chollet, François. 2017. Deep Learning with Python. Shelter Island: Manning. Economist, The. 2016. “From Not Working to Neural Networking.” The Economist Special Report, June 23, 2016. https://oreil.ly/6VvlS. Gerrish, Sean. 2018. How Smart Machines Think. Cambridge: MIT Press. Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. Cam‐ bridge: MIT Press. http://deeplearningbook.org.

Convolutional Neural Networks

|

445

Index

A

absolute risk aversion (see ARA) accuracy ratio, 162 action space, CartPole environment, 253 action-value policy, in QL, 260 activation functions, neural networks, 20, 425 AFI (artificial financial intelligence), 396-403 agent, in RL, 250 agents, 250 (see also algorithmic trading) DQLAgent, 260-270 FQLAgent, 271-276, 304-307, 335 Monte Carlo simulation, 255-257 preferences in normative finance, 67 QL agent, 33, 260-270, 357-363, 372 AGI (artificial general intelligence), 45 Agrawal, Ajay, 303 AI (artificial intelligence), 1 (see also AI-first finance) AI-based competition in finance, 379-391 algorithms, 3-9 forms of intelligence, 44 importance of data, 22-29 importance of hardware for progress in, 42-44 machine learning (see ML) neural networks in, 9-22 risks, regulation, and oversight issues, 388-391 SI and, 49-50 success stories, 32-42 AI-first finance, 185-205 efficient markets hypothesis, 186-191 financial singularity, 395-403

intraday market prediction, 204-205 market prediction with more features, 199-203 returns data as basis for market prediction, 192-198, 240-242 utopian versus dystopian outcomes, 54-56, 403 ALE (Arcade Learning Environment), 33, 38 algorithmic trading data retrieval, 347-350 deployment, 364-368 event-based backtesting, 357-364, 369-372 execution, 345-364 lack of research standardization, 382-383 Oanda environment, 345-346, 356-364, 369-373 risk management (see risk management) strategies, 280 vectorized backtesting (see vectorized back‐ testing) algorithms, generally, 3-9, 385 Allais paradox, 119-120, 121 alpha (above-market returns), 281, 379 AlphaGo algorithm, 38-39, 42-44 AlphaGo Zero (AlphaZero), 39-40, 41-42, 44, 46 alternative data sources, 113-117, 385 ambiguity in decision making, 118, 120-121 ANI (artificial narrow intelligence), 44, 49, 396 approximation task (see estimation task) APT (arbitrage pricing theory), 90-95 basic assumption of, 100 in data-driven finance, 134-143 risk measures and, 64-66, 90-95, 324

447

ARA (absolute risk aversion), 69 Arcade Learning Environment (see ALE) Arrow-Pratt absolute risk aversion, 69 artificial financial intelligence (see AFI) artificial general intelligence (see AGI) artificial intelligence (see AI) artificial narrow intelligence (see ANI) ask price, 351 Atari, 32-38 atomic outcome of technological singularity, 55 ATR (average true range) risk measure, 318-321 Augmented Dickey-Fuller test, 191 axiomatic theory, EUT as, 66-67 axioms, in finance, 66, 119

B

backtest() function, 308, 362, 372 backtesting risk measures, 288, 322-331 (see also event-based backtesting; vectorized backtesting) BacktestingBase class, 311-313, 322, 339 BacktestingBaseRM class, 322 backward propagation, shallow neural network estimation, 419-420 bagging method to avoid overfitting, 202, 204, 225-227 BaggingClassifier class, 202 bear strategy, AFI advantages for, 397 Bernoulli utility, 68 Bernoulli, Daniel, 118 BERT (bidirectional encoder representations from transformers) model, 381 bias and variance, in ML, 178-180 bias risk of AI in finance, 389 bid price, 351 big data, 28, 122, 381 biological enhancements, superintelligence, 46 Black-Scholes-Merton option pricing model, 382, 390 Bostrom, Nick, 39, 45, 51-54 boxing approach, SI control, 53 brain-machine hybrids, 47 bull strategy, AFI advantages for, 397, 398 buy market order, placing, 351

C

.calculate_net_wealth() method, 312 capability controls over SI, 53

448

|

Index

capacity of model or algorithm, in ML, 169-171, 182 capital market equilibrium, 83 capital market line (see CML) capital market theory (see CMT) CAPM (capital asset pricing model), 82-90 versus APT, 92, 136, 140 basic assumption of, 100 in data-driven finance, 130-134 EMH and, 191 versus EUT, 87 linear relationship assumption and, 153-155 risk measures and, 82-90, 324 CartPole game, 251-255 DQL agent, 260-264 versus Finance environment, 271 Monte Carlo simulation, 255-257 neural network approach, 33-38, 257-260 CFD (contracts for difference) trading, 346 chat bots, 381 chess, 40-42, 47 classification task, 8 AI’s ability to adjust to, 382 neural networks applied to, 20-22, 214, 244, 413-416, 421-423 .close_out() method, 312 cloud options for AI hardware, 44 clustering algorithms, 5 CML (capital market line), 83-90 CMT (capital market theory), 90 CNNs (convolutional neural networks), 33, 439-444 cognitive enhancement, SI goal of, 51 computer vision, and CNNs, 439 contracts for difference (CFD) trading (see CFD) convolutional neural networks (see CNNs) covariance matrix, MVP theory, 74 Cox-Ross-Rubenstein binomial option pricing model, 390 create_model() function, 290 create_rnn_model() function, 238 credit scoring, applying AI to, 380 cross-validation to avoid overfitting, in ML, 180-182 customer service, applying AI to, 381

D

data, 4

(see also classification task; estimation task) big data, 28, 122, 381 in finance and use of AI, 381 larger data, 26-28 for neural networks, 22-29 resources for finance industry, 385 small data set, 23-26 training algorithms, 172, 174-176, 178-182 volume and variety in prediction, 28 Data, News, Analytics platform (see DNA) data-driven finance, 99-155 data availability, 104-117 debunking central assumptions, 143-155 financial econometrics and regression, 100-104 normative theories and, 117-143 scientific method, 100 Deep Blue, 41 deep hedging, 380 deep learning (see DL) deep RNNs, 245 DeepMind, 32-40, 42-44 dense neural networks (see DNNs) dependent values, 10 deployment of trained trading bot, 364-368 derivatives hedging, applying AI to, 380 dimensionality reduction, CartPole environ‐ ment, 255 direct reward, in QL, 260 directed risk measures, financial instruments, 318 direction as binary feature, 200 discounted value, in QL, 260 distributed learning, bagging as, 227 DL (deep learning), 9 (see also Keras deep learning package) convolutional neural networks, 33, 439-444 dense neural networks (see DNNs) finding resources for, 385 predictive performance based on returns data, 192 recurrent neural networks (see RNNs) DNA (Data, News, Analytics) platform, 112-113 DNNs (dense neural networks), 211-228 bagging method to avoid overfitting, 225-227 baseline prediction, 214-218 data for example, 212-213

dropout, 220-222, 224 estimation task, 14-19 market efficiency, 195, 197, 200-205 normalization, 193, 218-220 optimizers, 227-228 regularization, 222-225 small data set, 25 vectorized backtesting, 289-301, 308-311 DQLAgent, in RL, 260-270 dropout, managing DNNs, 220-222, 224 RNNs, 245

E

econometrics, 101 economic inefficiencies, 192 (see also algorithmic trading) efficient frontier, 81 efficient market hypothesis (see EMH) efficient versus inefficient portfolio, MVP theory, 81 Eikon Data API, 105-108, 110, 212 Ellsberg paradox, 120-121 EMH (efficient market hypothesis), 186-191 environment, in RL, 250 EOD (end-of-day) versus intraday data, 110, 203 episode, in RL, 251 estimation task, 8 evaluation of algorithm, 172-177 machine learning example, 165-171 neural networks applied to, 14-19, 167-171, 243-244, 409-413, 417-420 OLS regression, 165-167 European call option, 64 EUT (expected utility theory), 66-71 basic assumption of, 100 in data-driven finance, 118-122 versus MVP and CAPM, 72, 87 risk measures and, 118, 324 event-based backtesting, 311-318 Oanda environment, 357-364, 369-372 transaction costs, 315-318 excess return of portfolio, MVP theory, 74 execution of trading algorithms account details, 356 applying AI to, 380 historical transactions, 356 order execution, 351-357

Index

|

449

trading bot learning, 357-363 expected returns under APT, 134-143 under CAPM, 130-134, 153-155 under MVP theory, 72, 129 expected utility theory (see EUT) expected variance of portfolio, MVP theory, 74 expected volatility in MVP theory, 74 exploitation, in RL, 258 exploration, in RL, 258

F

Fama, Eugene, 186 features data type, 4 Finance environment class, 265-276, 304, 333 finance industry AFI implications for, 396-403 AI-first finance (see AI-first finance) areas of AI application, 380-382 competitive scenarios for AI, 387-388 data-driven finance (see data-driven finance) education and training, 383-384 lack of AI standardization in, 382-383 market impact of AI, 386 normative finance (see normative financial theories and models) orthogonal skills and resources, 401 resource competition, 385-386 risks, regulation, and oversight issues for AI, 388-391 trading bots (see algorithmic trading) financial AI agents (see algorithmic trading) financial econometrics and regression, 100-104 financial market setting for RL, 264-270 financial price series, with RNN, 237-240 financial singularity, 395-403 financial time series data, 108 (see also market prediction) .fit_generator() method, 232 .fit() method, definition, 427 Fontaine, Philippe, 121 foreign exchange (see FX) forward propagation, shallow neural network estimation, 417 FQLAgent, 271-276, 304-307, 335 fraud detection, applying AI to, 380 FX (foreign exchange), 212-213, 237-240, 321, 346

450

|

Index

G

Gaussian normalization, 193, 218-220 .get_date_price() method, 311 Go, game of, 38-40 goal-content integrity, SI goal of, 51 Google LLC and data search, 114

H

hardware advances in AI, 42-44 resource competition, 386 hedge funds, AFI advantage for, 402 herding risk of AI in finance, 389 high dimensionality of data, AI’s ability to adjust to, 381 historical transactions, Oanda v20 API, 356 human resources (financial technology experts), 385 human traders, role in AFI, 401 humans, improving cognitive and physical per‐ formance of, 46 hyperparameters for neural network capacity, 170-171

I

implicitly assumed normal distribution, 72 incentives, SI control, 53 incremental learning characteristic of neural networks, 14, 22 independence axiom, 119 independent values, 10 indifference curves, risk-return space, 87-90 inexplicability risk of AI in finance, 389 instability of data, AI’s ability to adjust to, 381 instrumental goals of a superintelligence, 51-53 insufficient data risk of AI in finance, 390 intelligence, defining for AI, 44 intentional forgetting, training use of, 222 internet, as source of data for decision making, 122 intraday trading, 110, 204-205, 295-301

K

Kasparov, Garry, 40 Keras deep learning package, 15-22 DNNs (see DNNs) market prediction with neural network, 195 neural network implementations, 425

RNNs (see RNNs) Sequential model, 197, 200, 222, 227 KerasClassifier class, 225 KMeans algorithm, 5-6 know-how risk of AI in finance, 390

L

labels data type, 4 learning, 4-8, 162 (see also ML) leverage, ATR risk measurement in backtesting, 321 limit order, 353 linear activation function, 20, 425 linear algebra, 43 linear regression, in neural networks example, 11, 14 linear relationship assumption, 153-155 LSTM (long-short term memory) layer, 238, 244

M

machine intelligence (see AI) machine learning (see ML) margin stop out, 322 market orders backtesting risk measures, 319, 321, 324-331 execution of, 312, 351-357 market portfolios (see portfolios) market prediction with CNNs, 439-444 DNN with FX data, 211-228 expected returns (see expected returns) intraday market prediction, 204-205 with more features, 199-203 with neural networks, 175-177, 194-205, 435-437 with OLS regression, 177, 193, 196, 198 returns data as basis for, 192-198, 240-242 with RNNs, 240-242 with trading bots (see algorithmic trading) market risk, 85 Markov property of random walks, 187 maximum Sharpe ratio portfolio, 79 mean squared error (see MSE) mean-variance portfolio (MVP) theory (see MVP) .metrics() method, 427 microscopic alpha, 379

minimum volatility portfolio, 79 ML (machine learning), 9, 161-182 bias and variance, 178-180 capacity of model or algorithm, 169-171, 182 cross-validation, 180-182 deep learning, 9 evaluation of estimation algorithm, 172-177 finding resources for, 385 MSE of model or algorithm, 165-168 sample data set, 162-164 SL, 4, 122 MLP (multi-layer perceptron) (see DNNs) MLPClassifier model, scikit-learn, 25, 200 MLPRegressor class, scikit-learn, 14, 195, 197 model validation risk of AI in finance, 390 momentum feature, 199, 243 monopoly, AI-first finance landscape as, 387 Monte Carlo simulations, 39, 77-78, 255-257 Morgenstern, Oskar, 66-69 motivation selection methods, SI control, 53 MSE (mean squared error), 11, 162, 165-168, 174-177 multipolar outcome of technological singular‐ ity, 55 multivariate linear OLS regression, 92 Musk, Elon, 48, 391 MVP (mean-variance portfolio) theory, 72-82 basic assumption of, 100 versus CAPM, 83 in data-driven finance, 123-130 versus EUT, 72 quadratic utility function in, 87 risk measures and, 72-82

N

natural language processing (see NLP) ndarray objects, tensors represented as, 407 neural networks activation functions, 20, 425 in Atari story, 33-38 CartPole game, 33-38, 257-260 class-based implementations, 425-437 in classification task, 20-22, 214, 244 CNNs, 33, 439-444 cross-validation, 181 deep learning, 9 dense neural networks (see DNNs) in estimation task, 14-19, 167-171, 243-244

Index

|

451

market prediction with, 175-177, 194-205, 435-437 prediction algorithms and data size, 22-29 Q-Learning, 261-264 recurrent neural networks (see RNNs) role in AI, 9-22 shallow, 417-423, 431-437 simple, 409-416, 426-431 tensors and tensor operations, 407-409 whole brain emulation and, 48 Neuralink, 48 NLP (natural language processing), 110, 156 NNAgent class, 258 noise, in financial instrument price movements, 321 non-directed risk measures, financial instru‐ ments, 318 non-linearity of relationships, AI’s ability to adjust to, 381 non-normality of data, AI’s ability to adjust to, 381 nondiversifiable risk, 85 normalization, DNNs, 193, 218-220 normally distributed returns, 72, 87, 143-152, 187, 322 normative financial theories and models, 61-95 arbitrage pricing theory, 64-66, 90-95, 100, 134-143, 324 capital asset pricing model (see CAPM) versus empirical approaches to finance, 100 expected utility theory (see EUT) mean-variance portfolio theory, 72-82, 83, 87, 100, 123-130 uncertainty and risk, 62-66

O

Oanda trading platform, 108-110, 345 Oanda v20 API (Oanda environment) account, 346, 356 data retrieval, 347-350 deployment of trading bot, 364-368, 369-373 order execution, 351-357 risks and disclaimers, 346 trading bot training, 357-364 OandaEnv class, 357-362, 369-372 OandaTradingBot class, 364, 373 observation space, CartPole environment, 252 oligopoly, AI-first finance landscape as, 387

452

|

Index

OLS (ordinary least-squares) regression bias and variance in regression fit, 179 capacity in ML, 169 CAPM versus APT, 92 cross-validation, 181 efficient markets test, 187-191 estimation task learning, 165-167 as financial econometrics tool, 101-104 market prediction with, 177, 193, 196, 198 versus neural networks, 9-22, 236, 240 .on_success() method, 366 one-dimensional convolutional layer, 442 OpenAI Gym environment, 33-38, 251-255, 264-268 optimizers, DNNs, 227-228 ordinary least-squares (OLS) regression (see OLS) orthogonality hypothesis, 401 overfitting of data, avoiding, 174, 178-180, 202, 219-227, 245

P

paradoxa, in decision-making theory, 118-121 pattern recognition, AI algorithms’ focus on, 23 PDF (probability density function), 145-147 perfect competition, AI-first finance landscape as, 387 perfect financial economy, 63 .place_buy_order() method, 312 .place_sell_order() method, 312 policy, in RL, 251 portfolios applying AI to management of, 381 in CAPM theory, 83 CML, 83-90 MVP theory, 72-82, 83, 87, 100, 123-130 positive but decreasing marginal utility, 119 positive versus normative financial theory, 61 .predict() method, 427 prediction algorithm, 22-29 prediction of economic behavior data-driven (MVP theory), 122-130 physics envy and challenge of, 205-207 prediction, ML and, 172 (see also market prediction) .print_balance() method, 312 .print_net_wealth() method, 312 privacy risk of AI in finance, 388 probability, 62

probability density function (see PDF) probability measure, 63 probability space, 63 problem-agnostic characteristic of neural net‐ works, 22 programmatic APIs, data-driven finance, 108

Q

Q-Q (Quantile-Quantile) plot, 147, 150 QL (Q-learning), 33 (see also risk management) DQLAgent, 260-270 FQLAgent, 271-276, 304-307, 335 as trading bot, 357-363, 372 quadratic utility, normative financial theories, 87

R

random sample data set, OLS versus neural net‐ works, 17-19 random strategy, AFI advantages for, 397 random variable, 63 random walk hypothesis (see EMH) randomized sampling, in ML, 174 Refinitiv Workspace, 105, 212 regression task (see estimation task) regularization, DNNs, 222-225 regulation and oversight of AI in finance, 390, 390, 402 reinforcement learning (see RL) relu (rectified linear unit) function, 425 resource acquisition, SI goal of, 52 returns, 72 (see also expected returns) alpha (above-market returns), 281, 379 basing market prediction on, 192-198, 240-242 excess return of portfolio, 74 normally distributed, 72, 87, 143-152, 187, 322 revealed preferences, 122 reward, in RL, 251 risk and uncertainty (see uncertainty and risk) risk aversion, normative finance, 69 risk management, algorithmic trading, 303-331 assessing risk, 318-322 backtesting Python code, 333-342 backtesting risk measures, 322-331 DNN-based vectorized backtesting, 308-311

event-based backtesting, 311-318 FQLAgent trading bot, 304-307, 335 risk premium of portfolio, MVP theory, 74 risks of AI in finance, 388-390 RL (reinforcement learning), 4, 249-276 Atari story, 32-38 chess story, 41-42 DQLAgent, 260-270 example algorithm, 6-8 FQLAgent, 271-276 fundamental notions, 250-251 hardware advances and, 42-44 Monte Carlo agent, 255-257 neural network agent, 257-260 OpenAI Gym, 251-255 RNNs (recurrent neural networks), 229-246 dropout, 245 financial features, 242-246 financial price series, 237-240 financial return series, 240-242 first example, 230-234 second example, 234-236 rolling maximum feature, 199 rolling minimum feature, 199 rolling volatility feature, 199, 243 Ross, Stephen, 90 RWH (random walk hypothesis) (see EMH)

S

scientific method, 100 scikit-learn package, 14, 425 BaggingClassifier class, 202 KMeans algorithm, 5-6 market prediction on returns data, 194 MLPClassifier model, 25, 200 MLPRegressor class, 14, 195, 197 Sedol, Lee, 39 self-preservation, SI goal of, 51 sell market order, placing, 352 semi-strong form of EMH, 186 Sequential model, Keras package, 197, 200, 222, 227 set_seeds() function, 290 shallow neural networks, 417-423, 431-437 shann class, market prediction, 436-437 Sharpe ratio, 74, 79, 125-127, 129 shnn class, 433-437 SI (superintelligence), 45-56 versus AFI, 403

Index

|

453

artificial intelligence as path to, 49-50 biological enhancements, 46 brain-machine hybrids, 47 controls on, 53 goals of, 51-53 networks and organizations, 46 potential outcomes of, 54-56 reproduction or expansion of, 50 whole brain emulation, 48 sigmoid activation function, 20-22, 413, 421 simple moving average (see SMA) simple neural networks, 409-416, 426-431 SimpleRNN layer, 232, 238 singleton outcome of technological singularity, 55 sinn classification, 429-431 SL (supervised learning), 4, 122, 162, 255 (see also DNNs; RNNs) SMA (simple moving average), 199, 282-288, 293 societal impact of AFI, 402 softplus function, 425 St. Petersburg paradox, 118-119 state space, 62 state, in RL, 250 static economy, 62 stationarity of time series data, assumption of, 191 statistical inefficiencies DNNs, 211-228 predicting markets with neural networks, 192 RL, 249-276 RNNs, 229-246 statistical learning, 10, 101-103 step, in RL, 250 Stockfish chess engine, 41 stop loss (SL) order backtesting risk measures, 319, 321, 324-326, 330 execution of, 353 strong form of EMH, 186 structured streaming data, 108-110 stunting, SI control, 53 stylized facts, 150 superintelligence (see SI) supervised learning (see SL) Sutton, Richard, 251 symmetric information, economy, 63

454

|

Index

systemic risk, 85

T

take profit (TP) order backtesting risk measures, 319, 321, 328-331 execution of, 355 target, in RL, 251 TBBacktester class, 313-318 TBBacktesterRM class, 323, 342 technical indicators, adding to market predic‐ tion, 199-203 technological perfection, SI goal of, 51 technological singularity, 31, 50, 55 (see also SI) TensorFlow, 425 tensors and tensor operations, 407-409 test data set, training algorithm, 173, 177 TimeseriesGenerator (RNN), 231, 235 tpqoa wrapper, 346 trade execution (see execution of trading algo‐ rithms) traded assets, in static economy, 64 trading bots (see algorithmig trading) TradingBot class, 304, 335 trailing stop loss (TSL) order backtesting risk measures, 319, 326-328, 330 execution of, 354 training algorithms (see ML) training data set, training algorithm, 172, 174-176, 178-182 transaction costs event-based backtesting, 315-318 vectorized backtesting, 288, 298-301, 318 transparency risk of AI in finance, 390 tripwires, SI control, 54 Twitter API, 115-117 two fund separation theorem, 83

U

UL (unsupervised learning), 5-6, 162 uncertainty and risk, 62 (see also risk management) arbitrage pricing theory, 64-66, 90-95, 324 capital asset pricing model, 82-90, 324 expected utility theory, 66-71, 118, 324 mean-variance portfolio theory, 72-82 univariate linear OLS regression, 92 universal approximation characteristic of neu‐ ral networks, 22

unstructured data AI’s ability to adjust to, 382 historical, 110-111 streaming, 112-113 unsupervised learning (see UL) uploading (see WBE) utility functions, normative finance, 68 utopian versus dystopian perspectives on AI, 54-56, 403

V

validation data set, training algorithm, 172, 174-176, 178-182 vanishing alpha risk of AI in finance, 389 VaR (value-at-risk) risk measure, 318 vectorized backtesting, 281-301 DNNs, 289-301, 308-311 Oanda environment, 361-364, 372 SMA-based, 282-288 transaction costs, 288, 298-301, 318

versions of life, Tegmark’s, 47 volatility ATR risk measure, 318-321 expected volatility in MVP theory, 124-129 minimum volatility portfolio, 79 rolling volatility feature, 199, 243 von Neumann, John, 66-69

W

WBE (whole brain emulation), 48 weak form market efficiency, 186, 192, 198, 205 weights in neural network, 14, 222, 225

X

X% AFI success characteristic, 399

Z

z-score normalization, 193

Index

|

455

About the Author Dr. Yves J. Hilpisch is founder and managing partner of The Python Quants, a group focusing on the use of open source technologies for financial data science, artificial intelligence, algorithmic trading, and computational finance. He is also founder and CEO of The AI Machine, a company focused on AI-powered algorithmic trading via a proprietary strategy execution platform. In addition to this book, he is the author of the following books: • Python for Algorithmic Trading (O’Reilly, 2020) • Python for Finance (2nd ed., O’Reilly, 2018) • Derivatives Analytics with Python (Wiley, 2015) • Listed Volatility and Variance Derivatives (Wiley, 2017) Yves is an adjunct professor of computational finance and lectures on algorithmic trading at the CQF Program. He is also the director of the first online training pro‐ grams leading to university certificates in Python for Algorithmic Trading and Python for Computational Finance. Yves wrote the financial analytics library DX Analytics and organizes meetups, con‐ ferences, and bootcamps about Python for quantitative finance and algorithmic trad‐ ing in London, Frankfurt, Berlin, Paris, and New York. He has given keynote speeches at technology conferences in the United States, Europe, and Asia.

Colophon The animal on the cover of Artificial Intelligence in Finance is a bank vole (myodes glareolus). These voles can be found in forests, banks, and swamps throughout Europe and Central Asia, with notable populations in Finland and the United Kingdom. Bank voles are small, only 10-11 cm in length and 17-20 g on average, and have small eyes and ears. Their fur is thick and typically brown or gray, and covers their whole body. Bank voles have short tails and small brains relative to their body size. Pups are born blind and helpless in litters of four to eight, after after which they mature rather quickly, with females reaching maturity in two to three weeks and males maturing at six to eight weeks. The average lifespan for a bank vole mirrors this quick maturation, with most individuals living one half to two years. These small rodents are primarily active during twilight, though they can be diurnal or nocturnal as well. They have an omnivorous diet consisting mostly of plant matter, and what they eat changes with the seasons. Socially, female bank voles are dominant over males, the latter dispersing once reaching maturity, while female bank voles will typically stay closer to where they were born. Given their relatively healthy population numbers and wide distributions, the bank vole’s current conservation status is that of “Least Concern.” Many of the animals on O’Reilly covers are endangered; all of them are important to the world. The cover illustration is by Karen Montgomery, based on a black and white engraving from British Quadrupeds. The cover fonts are Gilroy Semibold and Guardian Sans. The text font is Adobe Minion Pro; the heading font is Adobe Myriad Condensed; and the code font is Dalton Maag’s Ubuntu Mono.