The Kalman Filter [1] originally by R. E. Kalman, is an algorithm used in control systems and signal processing to estimate the state of a system over time from a series of noisy measurements. The filter works in two steps: prediction and update. The prediction step uses the current state and a mathematical model of …
Continue reading “Using the Unscented Kalman Filter on Stock Prices”
Non-stationary data is a type of time series data where the statistical properties such as mean, variance, and autocorrelation change over time. In other words, the data exhibits some form of systematic change over time rather than fluctuating around a constant mean or with a constant variance. Non-stationary data can take several forms, including: Trends: …
Continue reading “Understanding N-HiTS Time Series Prediction”
In this post we explore the Instruct Llama2 created by Sridykhan [1] who altered Karpathy’s Baby Llama [2] model to “follow instructions and write tiny stories accordingly.” This new Instruct Llama2 model uses the same model and dataset as the original used by Karpathy. However, with Instruct Llama2, the model is trained with different inputs …
In this post we explore how to train the Llama2 model using the Baby Llama2 created by Andrej Karpathy [1] which is based on his original minGPT model [2] and has the same basic transformer architecture that other generative AI models use, such as ChatGPT [3]. Generative transformer models employ a stack of transformer blocks …
Continue reading “Understanding Baby Llama2 Training – A Visual Design Walkthrough”
ChatGPT is an amazing technology that has taken the world by storm. Under the hood, a highly trained large language model (LLM) creates the response to each query sent to the service. In February 2023 Meta released the open-source Llama2 LLM and on September 29, 2023, Meta released an open-source Llama2-Long LLM [1] which appears …
Continue reading “Understanding Llama2.c And ChatGPT Inferencing – A Visual Design Walkthrough”
In this blog post we discuss a MyCaffe implementation design of the paper “LoRA: Low-Rank Adaptation of Large Language Models” by Hu et al. [1] and describe how LoRA helps leverage the knowledge of the trained LLM to solve new specific problems in an efficient manner through fine-tuning. LLMs are immensely powerful but are created at …
Continue reading “Understanding LLM Fine Tuning with Low-Rank Adaptation (LoRA)”
In this post we describe a method of calculating change points using gaussian processes as described in the paper “Slow Momentum with Fast Reversion: A Trading Strategy Using Deep Learning and Changepoint Detection” by Wood et. al. [1], published in 2021. In addition, we show how the change point synthetic data enhances the Momentum Transformer …
In this post we describe the Temporal Fusion Transformer based Momentum Rebalancing Transformer described in the paper, “Trading with the Momentum Transformer: An Intelligent and Interpretable Architecture” by Wood et. al. [1], published in 2022. The original code analyzed can be found on GitHub at [2]. Time-series momentum (TSMOM) strategies such as ‘buying the winners and …
Continue reading “Understanding TFT Momentum Rebalancing for High Sharpe Ratios”
As discussed in our previous post, Change Point Detection (CPD) is an important part of time-series analysis used in numerous fields such as Medicine, Aerospace, Finance, Business, Metrology and Entertainment. In this post, we expand our analysis of an adaptive, online algorithm for Change Point Detection based on the paper “Memory-free Online Change-point Detection: A …
Continue reading “Understanding Adaptive LSTM-Autoencoder Change Point Detection”
Change Point Detection (CPD) is an important field of time-series analysis that provides methods of detecting changes in mean, variance, and distribution structure within time-series data. It has many applications in different fields, such as: Medicine: Change point detection can help monitor the health condition of patients, detect anomalies in vital signs, diagnose diseases, and …
Continue reading “Understanding Contrastive Change Point Detection”
