Economic Modeling for Forecasting

Machine learning beats traditional forecasting methods in multi series forecasting. In one of the latest M forecasting competitions, the aim was to advance what we know about time series forecasting methods and strategies. Competitors had to forecast 40k+ time series representing sales for the largest retail company in the world by revenue: Walmart. These are the main findings: ▶️ Performance of ML Methods: Machine learning (ML) models demonstrate superior accuracy compared to simple statistical methods. Hybrid approaches that combine ML techniques with statistical functionalities often yield effective results. Advanced ML methods, such as LightGBM and deep learning techniques, have shown significant forecasting potential. ▶️ Value of Combining Forecasts: Combining forecasts from various methods enhances accuracy. Even simple, equal-weighted combinations of models can outperform more complex approaches, reaffirming the effectiveness of ensemble strategies. ▶️ Cross-Learning Benefits: Utilizing cross-learning from correlated, hierarchical data improves forecasting accuracy. In short, one model to forecast thousands of time series. This approach allows for more efficient training and reduces computational costs, making it a valuable strategy. ▶️ Differences in Performance: Winning methods often outperform traditional benchmarks significantly. However, many teams may not surpass the performance of simpler methods, indicating that straightforward approaches can still be effective. Impact of External Adjustments: Incorporating external adjustments (ie, data based insight) can enhance forecast accuracy. ▶️ Importance of Cross-Validation Strategies: Effective cross-validation (CV) strategies are crucial for accurately assessing forecasting methods. Many teams fail to select the best forecasts due to inadequate CV methods. Utilizing extensive validation techniques can ensure robustness. ▶️ Role of Exogenous Variables: Including exogenous/explanatory variables significantly improves forecasting accuracy. Additional data such as promotions and price changes can lead to substantial improvements over models that rely solely on historical data. Overall, these findings emphasize the effectiveness of ML methods, the value of combining forecasts, and the importance of incorporating external factors and robust validation strategies in forecasting. If you haven’t already, try using machine learning models to forecast your future challenge 🙂 Read the article 👉 https://buff.ly/3O95gQp

Predicting #financialmarket stress has long proven to be a largely elusive goal. Advances in artificial intelligence and #machinelearning offer new possibilities to tackle this problem, given their ability to handle large datasets and unearth hidden nonlinear patterns. In the BIS paper , the authors have developed a new approach based on a combination of a recurrent neural network (RNN) and a large language model. Focusing on deviations from triangular arbitrage parity (TAP) in the Euro-Yen currency pair, our RNN produces interpretable daily forecasts of market dysfunction 60 business days ahead. To address the “black box” limitations of RNNs, our model assigns data-driven, time-varying weights to the input variables, making its decision process transparent. These weights serve a dual purpose. First, their evolution in and of itself provides early signals of latent changes in market dynamics. Second, when the network forecasts a higher probability of market dysfunction, these variable-specific weights help identify relevant market variables that we use to prompt an LLM to search for relevant information about potential market stress drivers.  - Source Bank for International Settlements – BIS

Stock volatility prediction forecasts the degree of price variation in financial assets over a future period. It is important for portfolio optimization (balancing risk and return), risk management (hedging against adverse market moves), and option pricing (determining fair contract values). Accurate volatility forecasts enable investors to make informed decisions and protect capital, especially during turbulent market conditions. Traditional models include econometric approaches like GARCH (captures volatility clustering) and its variants (eg, GJR-GARCH for asymmetric shocks), the HAR-RV model (captures long-memory properties), and Realized GARCH (integrates intra-day measures). More recent deep learning methods include LSTM networks (capture long-term dependencies), Transformers (model global temporal relations), and hybrid models combining CNNs for spatial features with LSTMs for temporal learning. Vision-based approaches transform time series into 2D images (eg, scalograms, Gramian Angular Fields) analyzed by CNNs or Vision Transformers (ViTs). Current challenges that Stock Volatility Prediction models face include: • financial data’s nonlinearity and non-stationarity, which linear models like GARCH fail to capture • the difficulty of extracting multi-scale temporal-frequency structures from raw 1D time series • reliance on CNNs that excel at local features but struggle to capture global dependencies in time-frequency representations • loss of intra-day information when using only close-to-close volatility estimators To address the challenges highlighted above, the authors of [1] propose TF-ViTNet, which is a dual-path hybrid model. First, the Parkinson’s (high-low) volatility series is transformed into 2D scalogram images using Continuous Wavelet Transform (CWT). This captures both time and frequency information simultaneously, overcoming the limitations of 1D sequences. Second, instead of using a CNN, a ViT is employed to process these scalograms. ViT’s self-attention mechanism captures global spatio-temporal patterns across the entire image, which CNNs miss. The TF-ViTNet model uses a parallel architecture: a ViT pathway processes scalograms for global patterns, while a separate LSTM pathway processes numerical technical indicators for temporal trends. The 2 streams are fused only at the final stage. Experimental results show that TF-ViTNet consistently outperforms econometric and machine-/deep-learning baselines. On NASDAQ (more volatile), it achieves the highest R^2 (0.387), substantially outperforming the CNN-based parallel model TF-CNet (R^2= −0.095) and LSTM-only (R^2=0.223). On S&P 500, TF-ViTNet achieves the highest R^2 (0.436) versus HAR-RV (0.373) and CNN-LSTM (0.422). TF-ViTNet also maintains stable predictive power during high-volatility regimes (eg, 2011 crisis, 2020 pandemic) and shows statistically significant improvements over most benchmarks in annual tests. Link to the paper [1] in the comments.

Kronos is a new AI model from Tsinghua University built to understand financial candlestick data like a language. Trained on 12 billion records from 45 markets, it forecasts prices 93% better than leading models, cuts volatility errors by 9%, and creates more realistic synthetic market data. Traders can use it for forecasting, risk management, and testing strategies. Best part: the pre-trained model is open source on GitHub. 🔹 Performance Kronos lifts price-forecast RankIC by 93 percent versus the leading TSFM and 87 percent versus the best non-pre-trained model. Volatility MAE drops 9 percent. Synthetic K-line fidelity improves 22 percent in zero-shot tests. 🔹 Data scale Pre-trained on over 12 billion K-line records from 45 exchanges and seven frequencies. Learns cross-asset, cross-timescale representations from OHLCVA that transfer across forecasting, risk, and generative tasks without fine-tuning. 🔹 Tokenizer A specialized coarse-to-fine tokenizer with Binary Spherical Quantization discretizes market moves into hierarchical tokens. Sequential subtoken prediction beats concurrent prediction and continuous baselines in ablations with matched parameters. 🔹 Investment impact Backtests on China A-shares show the top AER and IR among baselines. Test-time sampling lets desks trade accuracy for compute. Averaging multiple rollouts raises IC and RankIC without retraining. How to use it: Alpha research. Start with zero-shot price or return forecasting on your universe. Rank by Kronos signals and run quick AER and IR checks. Volatility. Plug Kronos volatility forecasts into position sizing and options surfaces. Synthetic data. Use Kronos-generated K-lines for stress tests and data augmentation. Validate with TSTR before deployment. Inference scaling. Ensemble multiple sampled trajectories for higher stability around rebalance. Credits: Authors: Yu Shi, Zongliang Fu, Shuo Chen, Bohan Zhao, Wei Xu, Changshui Zhang, Jian Li. Institute for Interdisciplinary Information Sciences and Department of Automation, Tsinghua University.

🚀 Just Released: ML-Enhanced Stock Prediction using Ito's Lemma! 📈 I'm excited to share my latest project that bridges the gap between traditional financial mathematics and cutting-edge machine learning! 🔬 What makes this special? ✅ Combines Ito's Lemma (stochastic calculus) with LSTM neural networks ✅ Multi-head attention mechanism for complex temporal pattern recognition ✅ Predicts volatility and drift parameters dynamically using ML ✅ 20+ technical indicators including RSI, MACD, Bollinger Bands ✅ Monte Carlo simulations with confidence intervals ✅ Real-time 6-month forecasting with uncertainty quantification 📊 Key Results: 68.3% profit probability (ML) vs 61.2% (traditional) 2.15% reduction in prediction uncertainty Dynamic parameter estimation that adapts to market conditions 🛠 Tech Stack: PyTorch for deep learning architecture LSTM + Attention for sequence modeling Geometric Brownian Motion enhanced with ML predictions yfinance for real-time market data This hybrid approach demonstrates how mathematical finance and AI can work together to create more robust prediction models. The model doesn't just predict prices—it learns market dynamics and adjusts volatility/drift parameters in real-time. 🔗 Open Source: Available on GitHub with comprehensive documentation and examples! Disclaimer: This is for educational/research purposes only. Always consult financial professionals for investment decisions. #MachineLearning #QuantitativeFinance #StochasticCalculus #DeepLearning #PyTorch #FinTech #DataScience #LSTM #StockPrediction #OpenSource What do you think about combining traditional financial mathematics with modern ML techniques? Would love to hear your thoughts! 💭

𝗝𝗣𝗠𝗼𝗿𝗴𝗮𝗻 𝗧𝗮𝘂𝗴𝗵𝘁 𝗔𝗜 𝘁𝗵𝗲 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗼𝗳 𝗠𝗮𝗿𝗸𝗲𝘁𝘀 JPMorgan researchers built a foundation model that predicts the next trade event the way an LLM predicts the next word — and it transferred to foreign markets it had never seen. TradeFM is a 524-million-parameter model trained on 10.7 billion tokens drawn from more than 9,000 U.S. equities across 368 trading days. Instead of language, its vocabulary is market microstructure: timing, size, price depth, and direction, compressed into 16,384 composite trade event tokens. 𝗪𝗵𝗮𝘁 𝘁𝗵𝗲𝘆 𝗱𝗶𝗱:  • Trained on U.S. equity trade-flow data from February 2024 to September 2025  • Tested inside a simulated exchange where the model predicts trades in a continuous loop  • Evaluated across 9 stocks, 3 liquidity tiers, and 9 months of held-out data — then applied, without any adjustments, to China and Japan 𝗪𝗵𝗮𝘁 𝘁𝗵𝗲𝘆 𝗳𝗼𝘂𝗻𝗱:  • TradeFM matched real market patterns 2 to 3 times more closely than a standard baseline  • Japan uses batch auctions at the open. China imposes 10% daily price limits. Performance degraded only moderately on both — without retraining Arman Khaledian, a former quant at Millennium and now CEO of Zanista AI, said: "That's not a toy result. 𝗜𝘁 𝗺𝗲𝗮𝗻𝘀 𝘁𝗵𝗲 𝗺𝗼𝗱𝗲𝗹 𝗶𝘀 𝗽𝗶𝗰𝗸𝗶𝗻𝗴 𝘂𝗽 𝘀𝗼𝗺𝗲𝘁𝗵𝗶𝗻𝗴 𝗿𝗲𝗮𝗹 𝗮𝗯𝗼𝘂𝘁 𝗵𝗼𝘄 𝗺𝗮𝗿𝗸𝗲𝘁𝘀 𝘄𝗼𝗿𝗸 𝗮𝘁 𝗮 𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗮𝗹 𝗹𝗲𝘃𝗲𝗹." He called it "the most interesting market simulation paper I've seen in a while. But it's a long way from a trading desk." Paper 𝘛𝘳𝘢𝘥𝘦𝘍𝘔: 𝘈 𝘎𝘦𝘯𝘦𝘳𝘢𝘵𝘪𝘷𝘦 𝘍𝘰𝘶𝘯𝘥𝘢𝘵𝘪𝘰𝘯 𝘔𝘰𝘥𝘦𝘭 𝘧𝘰𝘳 𝘛𝘳𝘢𝘥𝘦-𝘧𝘭𝘰𝘸 𝘢𝘯𝘥 𝘔𝘢𝘳𝘬𝘦𝘵 𝘔𝘪𝘤𝘳𝘰𝘴𝘁𝘳𝘶𝘤𝘵𝘶𝘳𝘦 Authors Maxime Kawawa-Beaudan, Srijan Sood, Kassiani Papasotiriou, Daniel Borrajo, Manuela Veloso If you want to hear how investors, quants, and analysts are using AI on Wall Street, check out my newsletter AI Street. Full write-up in the first comment.

Traditionally, time series forecasting has been treated as a pure sequence modeling problem. But real-world numbers don't exist in a vacuum—they are driven by unstructured, volatile context like breaking news and global events. While specialized Time Series Foundation Models (TSFMs) excel at identifying numerical patterns, they are often blind to these critical textual signals. Our new work, Nexus, tackles this by framing forecasting as an agentic reasoning problem. We introduce a multi-agent framework that seamlessly integrates unstructured contextual information with numerical data to synthesize accurate, well-reasoned forecasts. Key Results: 📈 ✅ Strong Performance Gains: Evaluated on highly volatile datasets—including Zillow real estate metrics and stock market equities succeeding LLM knowledge cutoffs—Nexus consistently matches or outperforms state-of-the-art TSFMs and strong LLM baselines. 🧠 Multi-Agent Decomposition: Our architecture isolates macro- and micro-level temporal fluctuations into specialized reasoning stages, allowing agents to process complex, multimodal reality much like a human financial analyst would. 🚀 Explicit Reasoning Traces: Beyond just outputting a number, Nexus produces high-quality reasoning logs that explicitly show the why behind each forecast, greatly improving interpretability and trust in the system. 💡 We believe that delivering on the promise of AI agents means pushing the boundaries of how systems reason. Nexus proves that real-world forecasting extends well beyond simple numerical extrapolation. 🔗 Read the full paper here: https://lnkd.in/gSKK3_zE Authors: Sarkar Snigdha Sarathi Das, Palash Goyal, Mihir Parmar, Nanyun (Violet) Peng, Vishy Tirumalashetty, Chun-Liang Li, Rui Zhang, Jinsung Yoon, Tomas Pfister #AI #ArtificialIntelligence #MachineLearning #TimeSeries #AIagents #LLM #CloudAI #Research

Traditional HFT systems are structurally price-agnostic optimized for latency, inventory control, and spread capture. Directionality is often considered noise. But that boundary is fading. By blending medium-frequency signals those operating on 1–5 minute horizons into the microstructure layer, we steer passive flow toward statistically favorable outcomes. No need to cross spreads or sacrifice queue priority. Just soft biasing of reservation price, quote asymmetry, and inventory targets, all driven by predictive structure. This backtest reconstructs L2 books tick-by-tick and simulates fill probabilities using probabilistic queue models. There’s no market impact modeled by necessity but for small clips, the simulation closely approximates the real mechanics. It's realistic enough to evaluate how signal shapes flow, not just returns. I’ve put this strategy live today. The real test begins now seeing how these MFT-informed passive quotes behave under real market pressure. Results will unfold over the coming days. And for context: several major HFT hedge funds already run multi-frequency desks, routing predictive signals into execution engines. This is part of a broader convergence forecast meets fill logic.

Jim Simons never spoke publicly about his specific models during his lifetime, but those who worked at Renaissance Technologies have pieced together the mathematical DNA of the Medallion Fund. At its core was a deep appreciation for Markov processes systems where the future depends only on the present state, not the entire past. Simons, a former codebreaker and mathematician, understood that markets are not random walks but processes with short-term memory structures that could be modeled probabilistically. The Medallion team reportedly used hidden Markov models (HMMs) to identify latent regimes bull, bear, sideways, high-volatility, low-volatility that were not directly observable from price data alone. By estimating the transition probabilities between these hidden states, the fund could adjust its positioning before regime shifts became obvious to the rest of the market. The insight was profound: instead of predicting price direction directly, predict the probability of being in a specific market state, then trade accordingly. For quantitative researchers today, the lesson endures: the most successful quant funds are not those with the most complex neural networks but those with the most accurate models of state transition. The Markov assumption that market memory is finite is not a limitation; it is the foundation upon which practical forecasting is built. #JimSimons #RenaissanceTechnologies #MarkovChains #HiddenMarkovModels #QuantFinance #MarketRegimes

"All models are wrong, but some are useful." – George Box I'm pleased to share my recent research paper on forecasting Brent crude oil prices using machine learning and technical indicators. In this work, I designed a stacked ensemble model combining Random Forest, XGBoost, and a neural network, built to improve short-term price prediction. To ensure interpretability, I applied SHAP analysis, making the model not only accurate but also explainable. The paper demonstrates how applied machine learning can provide actionable insights in high-volatility markets, while also emphasizing the value of transparency in financial modeling. #CommodityTrading #Oil #MachineLearning #Finance #EnergyMarkets #TimeSeriesForecasting #Python #XGBoost #NeuralNetworks #QuantitativeResearch #SHAP #BrentCrude #TradingStrategy #MLinFinance