In this blog post, we evaluate from a programmer’s perspective, the PatchTST model described in “A Time Series is Worth 64 Words: Long-term Forecasting with Transformers” by Nie, et. al., 2022. The PatchTST is a transformer-based model for multivariate time-series prediction that separates the input data into ‘patches’ that are then fed into a standard …
Continue reading “Understanding the PatchTST Model for Time Series Prediction”
In this blog post, we evaluate from a programmer’s perspective, the FSNet described in “Learning Fast and Slow for Online Time Series Forecasting” by Pham et. al., 2022.[1] The authors of FSNet describe the model as inspired by “Complementary Learning Systems (CLS) theory” to provide “a novel framework to address the challenges of online forecasting” …
Continue reading “Understanding FSNets Learning Fast and Slow for Online Time Series Forecasting”
Our last few posts on the Closed-form continuous-time (CfC) Liquid Neural Networks as described by [1], [2] and [3], have focused on learning to detect the last value in the input sequence. In this post, we focus on detecting multiple values where the previous 82 values of a curve are used to detect the next …
Continue reading “Detecting Multiple Values with a CfC Liquid Neural Network”
In this blog post, we compare the Liquid closed-form continuous-time neural network (CfC), as described by [1], [2] and [3], to the Long-Short Term Memory (LSTM) neural network and a simple Linear neural network to see how each model learns to track a simple Sine curve over time. To keep things simple, each test uses …
Continue reading “Comparing Other Models to a CfC Liquid Neural Network”
In this post we explore the impact of using different activations in the Closed-form Continuous-time (CfC) Liquid Neural Network as described by [1], [2] and [3]. Currently the CfC Unit Layer supports the following five different activations: RELU, SILU, GELU, TANH and LECUN. As shown below, each of these activations have different profiles. The visualization …
Continue reading “Comparing Activation Functions in a CfC Liquid Neural Network”
In our latest release of the MyCaffe AI Platform, version 1.12.2.41, we now support Liquid Neural Networks as described in [1], [2], [3] and [4]. Liquid neural networks, first introduced by [1], are dynamic networks constructed “of linear first-order dynamical systems modulated via nonlinear interlinked gates,” resulting in models that “represent dynamical systems with varying …
Continue reading “MyCaffe now supports Liquid Neural Networks!”
Liquid neural networks, first introduced by [1], are networks constructed “of linear first-order dynamical systems modulated via nonlinear interlinked gates,” resulting in models that “represent dynamical systems with varying (i.e., liquid) time-constants coupled to their hidden state, with outputs being computed by numerical differential equation solvers.” The Closed-form Continuous-time Models (CfC) ‘are powerful sequential liquid …
Continue reading “Closed-form Continuous-time Liquid Neural Net Models – A Programmer’s Perspective”
In our latest release of the MyCaffe AI Platform, version 1.12.1.82, we now support Temporal Fusion Transformer (TFT) Models as described in [1] and [2]. These powerful models provide multi-horizon time-series predictions while outperforming DeepAR from Amazon, Deep State Space Models, MQRNN, TRMF, and traditional models such as ARIMA and ETS according to [1]. The …
Continue reading “MyCaffe now supports Temporal Fusion Transformer Models!”
In our last post, we looked at the organization of the data used by Temporal Fusion Transformer models as described in the Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting article by Lim et al. [1]. In this post we take a deeper dive into the architecture of the Temporal Fusion Transformer model and how …
Continue reading “Temporal Fusion Transformers – Model Data Flow”
Temporal Fusion Transformers, as described in the Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting article by Lim et al. [1], use a complex mix of inputs to provide multi-horizon forecasting for timeseries data. The first step in understanding these models is to understand the data inputs fed into the model and the predicted outputs …
Continue reading “Temporal Fusion Transformers – Data Organization”
