Data Machina #208
Transformers and Time Series. LLM Autonomous Agents. English PySparkAI. Machine Unlearning. RAG with Graphs. Private MPT-30b. Instruction Tuning. LLaVAR.
Transformers and Time Series, Anyone? My friend and I are sitting at The Gun’s garden bar facing the river, right across Canary Wharf. Hundred of years ago in this pub, Lord Horatio Nelson would secretly meet his mistress upstairs. Smugglers would trade goods downstairs, while they’d carefully watch out for The Revenue Men through a secret spy-hole that it’s still there.
My friend works for a big investment bank in CW. He’s a super brilliant quant, a heavy drinker, and a compulsive gambler. So I guess I’m in perfect good company sitting at the right proper pub to talk about financial times series and deep learning.
I ask my friend if he’s read this review on the latest Deep Learning techniques for financial time series forecasting. He is not impressed, and very unamused. Rather unabated, I provide him with another reading suggestion: this excellent review on Self-Supervised Learning for Time Series Analysis.
“ Please! don’t get me annoyed with DL for fin time-series, OK?” my friend yells. He suggests I read this blog post: Time-Series Forecasting: Deep Learning vs Statistics — Who Wins? I concede on the DL cons. But I counter-argue that DL -for whatever cultural reasons- haven’t been extensively explored in finance.
I mumble: ”And now, there is all this stuff happening on attention, transformers, LLMs…” But my friend ignores me, as I suspect he’s had some nasty experiences applying DL for fin time series. “Let’s watch the boats as they glide over the great Thames…Cheers to Heraclitus!” he shouts. Then he gulps his pint.
Transformers and long-term forecasting. IBM researchers recently updated PatchTST, which is an efficient transformer-based model for multivariate time series forecasting and self-supervised representation learning. Checkout paper, code and explainer: A Time Series is Worth 64 Words: Long Term Forecasting with Transformers.
PatchTST, a breakthrough in time series forecasting. Hassem, wrote a great post in which he explains why PatchTST is a breakthrough and how patching works. He also provides a practical guide to implementing PatchTST in Python, and a comparison of PatchTST against SoTA MLP models like N-BEATS and N-HiTS. Post: PatchTST — A Step Forward in Time Series Forecasting
The AutoFormer and time series forecasting: In this blog post, the researchers review the paper Are Transformers Effective for Time Series Forecasting? They provide empirical evidence that transformers are indeed effective for time series forecasting and introduce the Autoformer. Checkout post, code, papers: Yes, Transformers are Effective for Time Series Forecasting
LLMs and explainable time series forecasting: This is an interesting paper in which the researchers propose an instruction fine tuned, Open-LLaMA model that generates explainable forecasts and achieve reasonable performance, albeit relatively inferior in comparison to GPT-4. Paper: Temporal Data Meets LLM: Explainable Financial Time Series Forecasting.
New ways to apply NNs to time series anomaly detection. A group of researchers just came up with Precursor-of-Anomaly (PoA) detection. This is a new method that uses a neural controlled differential equation-based neural net. The researchers claim that POA efficiently detects future anomalies before they happen. Paper: Precursor-of-Anomaly Detection for Irregular Time Series.
A unified Dl/ ML framework for time series. There is so much stuff happening around DL, transformers, adapters… and time series that it’s hard to keep up and have one single place with the latest libraries, techniques, models… Checkout Aeon: a unified DL/ML framework for all things time series.
A book on Time-series and Deep Learning. A friend of a friend of a friend, sent me an obscure, heavily encoded link. Apparently if you click on the link you could get a book on ML/DL Recipes for Time Series
Have a nice week.
10 Link-o-Troned
the ML Pythonista
Deep & Other Learning Bits
AI/ DL ResearchDocs
El Robótico
data v-i-s-i-o-n-s
MLOps Untangled
AI startups -> radar
ML Datasets & Stuff
Postscript, etc
Tips? Suggestions? Feedback? email Carlos
Curated by @ds_ldn in the middle of the night.