Using MyCaffe AI Platform in real-time inferencing

The SignalPop Trading Studio is Windows Store App that provides short-term option traders with real-time analytics geared to help the trader better understand what the market is doing during each intra-day trading session.  Part of the analytics provided by the SignalPop Trading Studio include real-time, AI driven price directional predictions which when taken together give a better idea of the current market momentum.

Each trading day, the application often processes over 50 million quotes, depending on the market activity, and runs over 40 thousand data points through the AI models using the MyCaffe AI Platform.  When running on a high-end GPU, such as the NVIDIA RTX A6000, these 40 thousand data points can be processed in under 100 milliseconds whereas on a low-cost GPU, such as the NVIDIA 1050 TI this processing is completed in around 250 milliseconds.  Each trading day, the MyCaffe AI Platform processes over 1 billion data points.

This high-level design document shows the data flow that pulses throughout the application on a 1-2 second scan cycle which allows the many analytic aspects of the product to produce real-time results that are then displayed on the price charts used when trading.

System Overview

The data firehose enters the application from the data vendor data stream and flows through the application like a river.  Vendor API calls are used to collect more pinpointed data and execute trades when desired.

SignalPop Trading Studio Architecture

Other than the main application and the charting that it provides, there are three main subsystems used by the application: Data Input, Extensions and Trading Execution.

The following sections describe the responsibilities and steps taken by each of these sub-systems as the application operates.

Data Input

All data fed into the system is first sent to the Data Condenser which is responsible for organizing all data into time synchronized time-periods used by the application.

For example, a vendor’s data stream data is (1) continually received asynchronously by the Data Provider module for that vendor and (2) sent to the Data Condenser which organizes the data by symbol and time-period.  The synchronized data is then sent asynchronously to (3) the charts for display, and (4) the extensions for further processing.

Data Provider modules are plug-n-play where new data providers are easily supported if they have a streaming and API based capabilities that provide standard market data formats.  A special Data Provider called the SQL Data Provider is used by our team internally for back-testing, and model development.


Extensions are plug-n-play modules that provide special functionality to the application.  Each plug-in is given access to the user interface and active chart where the plug-in is free to draw and display information related to the plug-in.  Time synchronized data packets flow to each plug-in on each data cycle and calculations made by one plug-in are passed to the next.

The AI based plug-ins AI Momentum Plug-in (5) and AI Trend Plug-in (7) both use the MyCaffe AI Platform (6)(8) to provide real-time insights based on trained AI models.  For example, the AI Momentum Plug-in (5) uses the MyCaffe AI Platform to detect the current market direction and predict the likely short-term future market direction for five different short term time periods and displays this information in real-time in the AI Momentum display.

The AI Trend Plug-in (7) is currently experimental and uses AI to detect the current trend direction and strength.  We are currently working on expanding the functionality of this plug-in.

The Drawing Plug-in (9) allows users to draw trend, support and resistance lines on the current chart so that traders can map out the structure of the market.

The Indicators Plug-in (10) gives traders a more in-depth view of historical changes in buy/sell pressure accumulation, correlation, and strength.

Trade Execution

The Trading Plug-in (11) gives traders real-time quote data on the options of the underlying and the underlying itself and allows traders to execute trades which are then displayed on the current price chart.

All trade executions (14) are performed by the Trading Vendor used.  The Trade Engine (12) manages the process of directing the trading vendor associated with the Trade Provider module for that vendor, to execute the trade as directed by the user.

Auto trading (15) is currently experimental and only used internally by SignalPop to test the viability of one model or another before it is published in the application as part of the signals produced by the AI Trend Extension Plug-in.  The Auto trading module (16) uses the MyCaffe AI Platform to help make better and more profitable trading decisions.


Using an AI platform in real-world, real-time inference-based applications has several unique requirements for the platform must run fast yet produce reliable results.  The following suggestions may help meet these objectives.

  • Use a GPU; the modern GPU’s, such as those produced by NVIDIA are fast, reliable, and surprisingly low cost on the low end if your model fits on a GPU that has 4 GB of video memory or less. This is important for crypto currency mining typically requires 6 GB or more which can dramatically increase the cost of the GPU itself.  The SignalPop Trading Studio processes 40 thousand data points in around 250 milliseconds on an NVIDIA 1050TI which only has 4 GB of video memory and can be purchased for under $300.
  • Use the same Data Forming during Training and Inferencing; when training it is critical to use the same data forming software both during training and during real-time inferencing. This may seem obvious but subtle differences in time synchronization and even data calculations can cause very different results when running in real-time inferencing.
  • Always test on different data from different time frames; When testing, it is critical to only use data that is completely unseen during the training process. For example, if a given data input used during training contains a 10-period window of data, the testing data should not contain any of the data points that fall within the same data window.  Again, this may seem obvious but can give misleading results if not followed.
  • Focus on Detection, then slowly move into Prediction; Detection is a far easier problem to solve that prediction – so start there first. Once the data patters you seek to find are easily detected, then and only then try to see how predictive they are.  If you can’t detect the pattern, you most certainly will not be able to predict what happens after it.

In this post, we showed how an application can use AI to help empower traders with better insights that are produced by analyzing vast amounts of data in real-time.  Gone are the days where a trader needs 9 to 12 screens to analyze market direction – AI has the power to process all of that data and more in real-time and produce more consistent results than even the most seasoned trader.

Happy Deep Learning!