Generative AI for Trading and Asset Management

Table of contents PREFACE ACKNOWLEDGMENTS About the Authors PART I: GENERATIVE AI FOR TRADING AND ASSET MANAGEMENT: A LOW-CODE INTRODUCTION 1. NO-CODE GENERATIVE AI FOR BASIC QUANTITATIVE FINANCE 1.1 RETRIEVING HISTORICAL MARKET DATA 1.2 COMPUTING SHARPE RATIO 1.3 DATA FORMATTING AND ANALYSIS 1.4 TRANSLATING MATLAB CODES TO PYTHON CODES 1.5 CONCLUSION 2. NO-CODE GENERATIVE AI FOR TRADING STRATEGIES DEVELOPMENT 2.1 CREATING CODES FROM A STRATEGY SPECIFICATION 2.2 SUMMARIZING A TRADING STRATEGY PAPER AND CREATING BACKTEST CODES FROM IT 2.3 SEARCHING FOR A PORTFOLIO OPTIMIZATION ALGORITHM BASED ON MACHINE LEARNING. 2.4 EXPLORE OPTIONS TERM STRUCTURE ARBITRAGE STRATEGIES 2.5 CONCLUSION 2.6 EXERCISES 3. WHIRLWIND TOUR OF ML IN ASSET MANAGEMENT 3.1 UNSUPERVISED LEARNING 3.1.1 HIERARCHICAL RISK PARITY (HRP) 3.1.2 PRINCIPAL COMPONENT ANALYSIS (PCA) 3.1.3 CLUSTER-BASED FEATURE SELECTION (CMDA) 3.1.4 HIDDEN MARKOV MODEL (HMM) 3.2 SUPERVISED LEARNING 3.2.1 LINEAR AND LOGISTIC REGRESSIONS 3.2.2 L1 AND L2 REGULARIZATIONS 3.2.3 HYPERPARAMETER OPTIMIZATION, VALIDATION AND CROSS-VALIDATION 3.2.4 PERFORMANCE METRICS 3.2.5 CLASSIFICATION AND REGRESSION TREES, RANDOM FOREST, AND BOOSTED TREES 3.2.6 NEURAL NETWORKS 3.2.7 RECURRENT NEURAL NETWORK 3.3 DEEP REINFORCEMENT LEARNING 3.4 DATA ENGINEERING 3.4.1 UNIQUE COMPANY IDENTIFIERS 3.4.2 DIVIDEND AND SPLIT ADJUSTMENTS 3.4.3 SURVIVORSHIP BIAS 3.4.4 LOOK-AHEAD BIAS 3.5 FEATURES ENGINEERING 3.5.1 STATIONARITY 3.5.2 MERGING TIME SERIES WITH DIFFERENT FREQUENCIES 3.5.3 TIME-SERIES VS CROSS-SECTIONAL FEATURES 3.5.4 VALIDATING THIRD-PARTY FEATURES 3.5.5 GENERATIVE AI AS A FEATURE GENERATOR 3.5.6 FEATURES IMPORTANCE RANKING AND SELECTION 3.6 CONCLUSION PART II: DEEP GENERATIVE MODELS FOR TRADING AND ASSET MANAGEMENT 4. UNDERSTANDING GENERATIVE AI 4.1 WHY GENERATIVE MODELS 4.2 DIFFERENCE WITH DISCRIMINATIVE MODELS 4.3 HOW CAN WE USE THEM? 4.3.1 PROBABILITY DENSITY ESTIMATION 4.3.2 GENERATING NEW DATA 4.3.3 LEARNING NEW DATA REPRESENTATIONS. 4.4 TAXONOMY OF GENERATIVE MODELS 4.5 CONCLUSION 5. DEEP AUTO-REGRESSIVE MODELS FOR SEQUENCE MODELING 5.1 REPRESENTATION COMPLEXITY 5.2 REPRESENTATION AND COMPLEXITY REDUCTION 5.3 A SHORT TOUR OF KEY MODEL FAMILIES 5.3.1 LOGISTIC REGRESSION MODEL 5.3.1.1 Sampling from FVSN 5.3.2 MASKED AUTO ENCODER FOR DENSITY ESTIMATION (MADE) 5.3.3 CAUSAL MASKED NEURAL NETWORK MODELS 5.3.3.1 WaveNet 5.3.4 RECURRENT NEURAL NETWORKS (RNN) Practical Considerations and Limitations 5.3.5 TRANSFORMERS 5.3.5.1 Attention mechanism 5.3.5.2 Scaled Dot-Product Attention 5.3.5.3 From Self-Attention To Transformers 5.3.5.4 Positional Encodings 5.3.5.5 MultiHeaded Attention 5.3.5.6 The Feed-Forward Layer 5.3.5.7 Add & Norm blocks 5.3.5.8 The Transformer Encoder layer 5.3.5.9 The Complete Transformer Encoder 5.3.5.10 Model Objective. 5.3.6 FROM NLP TRANSFORMER TO THE TIME SERIES TRANSFORMERS 5.3.6.1 Discretizing Time Series Data: The Chronos Approach. 5.3.6.2 Continuous Input for Transformers: The Lag-Llama Approach 5.3.6.2.1 Innovations and approaches 5.4 MODEL FITTING 5.5 CONCLUSIONS 6. DEEP LATENT VARIABLE MODELS 6.1 INTRODUCTION 6.2 LATENT VARIABLE MODELS 6.3 EXAMPLES OF TRADITIONAL LATENT VARIABLE MODELS 6.3.1 FACTOR ANALYSIS 6.3.2 PROBABILISTIC PRINCIPAL COMPONENT ANALYSIS Example: Comparing PCA and Factor Analysis for Latent Space Recovery Advantages of Probabilistic Approaches (PPCA/FA) over PCA 6.3.3 GAUSSIANS MIXTURE MODELS 6.3.3.1 Gaussian Mixture Model (GMM) for Market Regime Detection Example: Low and High Volatility Regimes 6.3.4 DEEP LATENT VARIABLE MODELS 6.4 LEARNING 6.4.1 TRAINING OBJECTIVE 6.4.2 THE VARIATIONAL INFERENCE APPROXIMATION How to Choose the Proposal Distribution Amortized inference. 6.4.3 OPTIMIZATION 6.4.3.1 The Likelihood gradient, REINFORNCE 6.4.3.2 Reparameterization trick 6.4.4 MIND THE GAP! 6.5 VARIATIONAL AUTO ENCODERS (VAE) 6.6 VAES FOR SEQUENTIAL DATA AND TIME SERIES 6.6.1 EXTENDING VAES FOR TIME SERIES Sequential Encoders and Decoders Superposition of Time Serie Components 6.6.1.1 TimeVAE: A flexible VAE for Time Series Generation Architecture of TimeVAE 6.7 CONCLUSION 7. FLOWS MODELS 7.1 INTRODUCTION 7.2 TRAINING 7.3 LINEAR FLOWS 7.4 DESIGNING NON LINEAR FLOWS 7.5 COUPLING FLOWS 7.5.1 NICE: NONLINEAR INDEPENDENT COMPONENTS ESTIMATION 7.5.2 REAL-NVP: NON VOLUME PRESERVING TRANSFORMATION 7.6 AUTOREGRESSIVE FLOWS 7.7 CONTINUOUS NORMALIZING FLOWS 7.8 MODELING FINANCIAL TIME SERIES WITH FLOW MODELS. 7.8.1 TRANSITIONING FROM IMAGE DATA TO TIME SERIES DYNAMICS 7.8.2 ADAPTING FLOWS FOR TIME SERIES. 7.8.3 CASE STUDY: A PRACTICAL EXAMPLE - CONDITIONED NORMALIZING FLOWS 7.8.3.1 Importance of Domain Knowledge in Financial Time Series 7.9 CONCLUSION 8. GENERATIVE ADVERSARIAL NETWORKS 8.1 INTRODUCTION 8.2 TRAINING 8.2.1 EVALUATION 8.3 SOME THEORETICAL INSIGHT IN GANS 8.4 WHY IS GAN TRAINING HARD? IMPROVING GAN TRAINING TECHNIQUES 8.5 WASSERSTEIN GAN (WGAN) 8.5.1 GRADIENT PENALTY GAN (WGAN-GP) 8.6 EXTENDING GANS FOR TIME SERIES 9. LEVERAGING LLMS FOR SENTIMENT ANALYSIS IN TRADING 9.1 SENTIMENT ANALYSIS IN FED PRESS CONFERENCE SPEECHES USING LARGE LANGUAGE MODELS 9.2 DATA: VIDEO + MARKET PRICES 9.2.1 COLLECTING AUDIO DATA 9.3 SPEECH TO TEXT CONVERSION 9.3.1 WHISPER MODEL 9.3.1.1 Python usage 9.3.2 WHISPER ON FED SPEECH AUDIO DATA 9.3.3 AUDIO SEGMENTATION 9.4 SENTIMENT ANALYSIS 9.4.1 BERT 9.4.1.1 BERT Overview Input/Output Representations Input Representations Output Representations Pre-training objectives 9.4.1.2 Fine-Tuning BERT for Enhanced Financial Sentiment Analysis: Producing FinBERT 9.4.1.3 Using FinBERT 9.5 EXPERIMENT RESULTS 9.6 CONCLUSION 10. EFFICIENT INFERENCE 10.1 INTRODUCTION 10.2 SCALING LARGE LANGUAGE MODELS: HIGH PERFORMANCE, HIGH COMPUTATIONAL COST, AND EMERGENT ABILITIES. 10.2.1 EMERGENT ABILITIES 10.2.2 IMPACT OF MODEL SIZE 10.2.3 EFFECT OF TRAINING TIME. 10.2.4 EFFICIENT INFERENCE FOR DEEP MODELS. 10.3 MAKING FINBERT FASTER 10.3.1 KNOWLEDGE DISTILLATION 10.3.1.1 Which aspect of the teacher model to match 10.3.1.2 Response-based knowledge. 10.3.1.3 Implementation details. 10.3.2 CASE STUDY RESULTS. DISTILLING FINBERT 10.4 MODEL QUANTIZATION 10.4.1 LINEAR QUANTIZATION 10.4.1.1 Example of Linear Quantization 10.4.2 QUANTIZING AN ATTENTION LAYER IN DISTILLED FINBERT 10.4.3 EXPERIMENT RESULTS WITH LINEAR QUANTIZATION ON DISTILLEDFINBERT 10.5 CUSTOMIZING YOUR LLM: ADAPTING MODELS TO YOUR NEEDS 10.5.1 FINE-TUNING TECHNIQUES. 10.5.1.1 Traditional Fine-Tuning (FT): 10.5.1.2 Parameter-Efficient Fine-Tuning (PEFT) BitFit Adapters Prompt-tuning LoRA (Low-rank adaptation of large language models) Efficiency of LoRA QLoRA 10.5.1.3 Aligning Your LLM with Human Preferences 10.6 CONCLUSIONS 11. AFTERWORD REFERENCES APPENDICES APPENDIX A — A.1 RETRIEVING ADJUSTED CLOSING PRICES AND COMPUTING DAILY RETURNS A.2 INSTALLING PYTHON A.2.1 STEP 1: DOWNLOAD PYTHON A.2.2 STEP 2: INSTALL PYTHON A.2.3 STEP 3: SET UP A VIRTUAL ENVIRONMENT (OPTIONAL BUT RECOMMENDED) A.2.4 STEP 4: INSTALL PACKAGES WITH PIP A.2.5 STEP 5: CONSIDER AN INTEGRATED DEVELOPMENT ENVIRONMENT (IDE) A.2.6 ADDITIONAL TIPS A.3 PLOTTING THE RISK-FREE-RATE OVER THE YEARS A.4 COMPUTING THE SHARPE RATIO OF SPY A.5 MATLAB CODE FOR COMPUTING EFFICIENT FRONTIER AND FINDING THE TANGENCY PORTFOLIO APPENDIX B — B.1 COMPUTING NEXT-DAY’S RETURN B.2 UPLOADING THE FAMA-FRENCH FACTORS B.3 COMBINING FAMA-FRENCH FACTORS WITH NEXT-DAY’S RETURNS References Index