Recurrent Learning Now Supported with cuDNN 7.4.1 on Char-RNN to learn Shakespeare!

In our latest release, version 0.10.0.122, we now support Recurrent Learning with both the LSTM [1] and LSTM_SIMPLE [2] layers to solve the Char-RNN as described by [3] and inspired by adepierre [4] and create a Shakespeare sonnet, and do so with the recently released CUDA 10.0.130/cuDNN 7.4.1.

The thought of his but is the queen of the wind:
Thou hast done that with a wife of bate are to the
earth, and straker'd them secured of my own to with the
more.

CORIOLANUS:
My lord, so think'st thou to a play be with thee,
And mine with him to me, which think it be the gives
That see the componted heart of the more times,
And they in the farswer with the season
That thou art that thou hast a man as belied.

PRONEES:
That what so like the heart to the adficer to
the father doth and some part of our house of the server.

DOMIONA:
What wishes my servant words, and so dose strack,
here hores ip a lord.

PARELLO:
And you are grace; and a singer of your life,
And his heart mistress fare the dear be readors
To the buse of the father him to the sone.

HOMITIUS ENOBARY:
And they are not a his wonders for thy greater;
But but a plotering pastice and every sirs.

PAPOLLES:
I will not as my lord, and the prince and house,
But that is scort to my wanter with her than.

To create the Shakespeare above, we used the Char-RNN model shown below with the LSTM Layer.

Char-RNN LSTM based Model

To solve this model, the SignalPop AI Designer uses the new MyCaffeTrainerRNN (which operates similarly to the MyCaffeTrainerRL) to train the recurrent network.

But what actually happens within each LSTM layer to make this happen?

Each LSTM layer contains an Unrolled Net which unrolls the recurrent operations.  As shown below, you can see that each of the 75 elements within the sequence are ‘unrolled’ into a set of layers: SCALE, INNER_PRODUCT, RESHAPE and LSTM_UNIT.  Each set process the data and feeds it into the next thus forming the recurrent nature of the network.

LSTM Unrolled Net

During the initialization (steps 1-3) the weights are loaded and data is fed into the input blob.  Next in steps 4-8, the recurrent nature of the network processes the each item within the sequence. Upon reaching the end the results are concatenated to form the output in steps 9 and 10.  This same process occurs in both LSTM layers – so the full Char-RNN actually has over 600 layers.  Amazingly all of this processing for both the forward and backward pass happens in around 64 milliseconds on a Titan Xp running in TCC mode!

To try out this model and train it yourself, just check out our Tutorials for easy step-by-step instructions that will get you started quickly! For cool example videos, including a Cart-Pole balancing video, check out our Examples page.

New Features

The following new features have been added to this release.

  • RTX 2080ti now supported!
  • CUDA 10.0.130/cuDNN 7.4.1/nvapi 410/driver 416.94.
  • Native Caffe updated through 10/24/2018.
  • Added new ClipLayer.
  • Added new LSTMLayer.
  • Added new RNNLayer.
  • Added new InputLayer.
  • Added new ReshapeLayer.
  • Added new MyCaffeGymTrainerRNN for RNN Training.
  • Added new Output Window for MyCaffeGymTrainerRNN output.
  • Added support for Char-RNN networks used on text files.
  • Added support for Char-RNN style networks used on audio WAV files.
Bug Fixes

The following bug fixes have been made in this release.

  • Fixed bugs in ATARI negative results not showing up in Project window.
  • Fixed bugs in running Test on gyms.

 


[1] J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko and T. Darrell, (2014), Long-term Recurrent Convolutional Networks for Visual Recognition and Description, Arxiv.
[2] junhyukoh, (2016), junkyukoh/caffe-lstm Github, GitHub.com.
[3] Karpathy, A., (2015), The Unreasonable Effectiveness of Recurrent Neural Networks, Andrej Karpathy blog.
[4] adepierre, (2017), adepierre/caffe-char-rnn Github, GitHub.com.

Policy Gradient Reinforcement Learning Now Supported with cuDNN 7.3.1 on an ATARI Gym!

In our latest release, version 0.10.0.76, we now support multi-threaded, Policy Gradient Reinforcement Learning on the Arcade-Learning-Environment [4] (based on the ATARI 2600 emulator [5]) as described by Andrej Karpathy[1][2][3], and do so with the recently released CUDA 10.0.130/cuDNN 7.3.1.

Using the simple Sigmoid based policy gradient reinforcement learning model shown below…

Simple Sigmoid based Policy Gradient RL Model for ATARI

… the SignalPop AI Designer uses the MyCaffeTrainerRL to train the model to play ATARI Pong better than ATARI!

To try out this model and train it yourself, just check out our Tutorials for easy step-by-step instructions that will get you started quickly! For cool example videos, including a Cart-Pole balancing video, check out our Examples page.

New Features

The following new features have been added to this release.

  • CUDA 10.0.130/cuDNN 7.3.1/driver 411.70 and 416.16 support added.
  • Added new ATARI Gym support.
  • Added random exploration support.
  • Added ability to turn on/off policy gradient accelerated learning.
Bug Fixes

The following bug fixes have been made in this release.

  • Fixed bugs related to accumulating gradients and models with null diffs.
  • Fixed bugs in discounted return calculation.
  • Fixed bugs in Blob.NormalizeData.
  • Fixed bugs in ATARI results not showing in Project Window (0.10.0.76)
  • Fixed bugs in running Test on gyms (0.10.0.76)
New Publications

 



[1] Karpathy, A., Deep Reinforcement Learning: Pong from Pixels, Andrej Karpathy blog, May 31, 2016.
[2] Karpathy, A., GitHub:karpathy/pg-pong.py, GitHub, 2016.
[3] Karpathy, A., CS231n Convolutional Neural Networks for Visual Recognition, Stanford University.
[4] The Arcade Learning Environment: An Evaluation Platform for General Agents, by Marc G. Bellemare, Yavar Naddaf, Joel Veness and Michael Bowling, 2012-2013. Source code available on GitHub at mgbellemare/Arcade-Learning-Environment.
[5] Stella – A multi-platform Atari 2600 VCS emulator by Bradford W. Mott, Stephen Anthony and The Stella Team, 1995-2018. Source code available on GitHub at stella-emu/stella

Softmax based Policy Gradient Reinforcement Learning Now Supported with CUDA 10!

In our latest release, version 0.10.0.24, we now support multi-threaded, SoftMax based Policy Gradient Reinforcement Learning as described by Andrej Karpathy[1][2][3], and do so with the recently released CUDA 10.0.130/cuDNN 7.3.

Using the simple SoftMax based policy gradient reinforcement learning model shown below…

Simple Softmax based Policy Gradient RL Model for Cart-Pole

… the SignalPop AI Designer uses the MyCaffeTrainerRL to train the model to solve the Cart-Pole problem and balance the pole.

To use the MyCaffeTrainerRL, just set the custom_trainer Solver property to RL.Trainer and you are ready to go.

To try out this model and train it yourself, just check out our Tutorials for easy step-by-step instructions that will get you started quickly! For cool example videos, including a Cart-Pole balancing video, check out our Examples page.

New Features

The following new features have been added to this release.

  • CUDA 10.0.130/cuDNN 7.3/driver 411.63 support added.
  • Added SoftMax support to Policy Gradient Reinforcement Learning.
  • Added multi-threading support to Policy Gradient Reinforcement Learning (across GPU’s).
Bug Fixes

The following bug fixes have been made in this release.

  • Fixed bug Policy Gradient accumulation, speeding up learning by 3x!
  • Fixed bug in snapshots related to Policy Gradient learning.


[1] Karpathy, A., Deep Reinforcement Learning: Pong from Pixels, Andrej Karpathy blog, May 31, 2016.
[2] Karpathy, A., GitHub:karpathy/pg-pong.py, GitHub, 2016.
[3] Karpathy, A., CS231n Convolutional Neural Networks for Visual Recognition, Stanford University.

Policy Gradient Reinforcement Learning Now Supported!

In our latest release, version 0.9.2.188, we now support Policy Gradient Reinforcement Learning as described by Andrej Karpathy[1][2][3], and do so with the recently released CUDA 9.2.148 (p1)/cuDNN 7.2.1.

For training, we have also added a new Gym infrastructure to the SignalPop AI Designer, where the dataset in each project can either be a standard dataset, or a dynamic gym dataset, such as the Cart-Pole gym inspired by OpenAI[4][5] (originally created by Richard Sutton et al. [6][7]).

Using a simple policy gradient reinforcement learning model shown below…

Simple Sigmoid based Policy Gradient RL Model for Cart-Pole

… the SignalPop AI Designer uses the new MyCaffeTrainerRL to train the model to solve the Cart-Pole problem and balance the pole.

To use the MyCaffeTrainerRL, just set the custom_trainer Solver property to RL.Trainer and you are ready to go.

To try out this model and train it yourself, just check out our Tutorials for easy step-by-step instructions that will get you started quickly!  For other cool example videos, check out our Examples page.

New Features

The following new features have been added to this release.

  • CUDA 9.2.148 (p1)/cuDNN 7.2.1/driver 399.07 support added.
  • Added support for Policy Gradient Reinforcement Learning via the new MyCaffeTrainerRL.
  • Added new Gym support via the new Gym dataset type along with the new Cart-Pole gym.
  • Added a new MemoryLoss layer.
  • Added a new SoftmaxCrossEntropyLoss layer.
  • Added a new LSTMSimple layer.
  • Added layer freezing to allow for each Transfer Learning.
Bug Fixes

The following bug fixes have been made in this release.

  • Fixed bug in LOAD_FROM_SERVICE (was not working)
  • Fixed bugs in Label Visualization.
  • Fixed bugs in Weight Visualization.
  • Fixed bugs related to Importing Weights.


[1] Karpathy, A., Deep Reinforcement Learning: Pong from Pixels, Andrej Karpathy blog, May 31, 2016.
[2] Karpathy, A., GitHub:karpathy/pg-pong.py, GitHub, 2016.
[3] Karpathy, A., CS231n Convolutional Neural Networks for Visual Recognition, Stanford University.
[4] OpenAI, CartPole-V0.
[5] OpenAI, GitHub:gym/gym/envs/classic_control/cartpole.py, GitHub, April 27, 2016.
[6] Barto, A. G., Sutton, R. S., Anderson, C. W., Neuronlike adaptive elements that can solve difficult learning control problems, IEEE, Vols. SMC-13, no. 5, pp. 834-846, September 1983.
[7] Sutton, R. S., et al., incompleteideas.net/sutton/book/code/pole.c, 1983.

Inroducing the SignalPop Universal Miner!

You can now easily mine Ethereum with our newly released SignalPop Universal Miner where all that you need to do is control your ambient temperature – we take care of the rest!

SignalPop Universal Miner

Built entirely for Windows bases systems (Windows 10 highly recommended), the Universal Miner carefully monitors your GPU’s and keeps their temperatures within a 5 degree Celsius range of the target temperature that you set.

Setting Temperature Targets

On hot days, we automatically ramp up the fans and on cold days we reduce them.

Active Hardware Monitoring

In addition to temperature control, the Universal Miner monitors the underlying mining software making sure that it continually runs.

Our goal is to keep you mining 24/7!

Mining for Charity

Whether operating a mining farm or running on a home computer, with the SignalPop Universal Miner you can optionally give a portion of your mining time to a charity.  During which time your mining work is done on behalf of the charity and your compensation is sent to their Ethereum address.

Mining for Charity

We give you a link to each charity so that you can vet them and decided which one’s you would like to donate your mining efforts to.

Monitor Your Account

The SignalPop Universal Miner uses the Ethermine Pool for all mining work – to view your account, simply select the ‘Account’ button which will take you directly to your account on their site where you can easily track your earnings.

You can get the free download from our Products page.

Happy Mining!

Deep Convolutional Auto-Encoders for MNIST Now Supported!

In our latest release, version 0.9.2.122, we now support deep convolutional auto-encoders with pooling as described by [1], and do so with the new ly released CUDA 9.2.148/cuDNN 7.1.4.

Auto-encoders are models that learn how to re-create the input fed into them.  In our example shown here, the MNIST dataset is fed into our model,…

Auto-Encoder Input

…a Deep Convolutional Auto-Encoder Model named AutoEncPool2.

Deep Convolutional Auto-Encoder Model with Pooling

The AutoEncPool2 model learns to re-create the MNIST inputs as shown below, which is the visualization of the last BATCHNORM output layer, bn5.

Auto-Encoder Learned Output

The magic occurs in the ip2encode layer which contains the 30 element encoding learned from the MNIST dataset.  To view the encoding, merely inspect the DEBUG layer attached to it.

Debug Layer

Inspecting the DEBUG layer produces a t-SNE analysis of the embedding that clearly shows data separation for each class within the learned encoding.

Auto-Encoder t-SNE Analysis

Such auto-encoder’s can be helpful in pre-training models and performing data reduction tasks.

If you would like to try the Auto-Encoder model yourself, just follow the easy steps in the Auto-Encoder tutorial.

References

[1] Volodymyr Turchenko, Eric Chalmers, Artur Luczac, A Deep Convolutional Auto-Encoder with Pooling – Unpooling Layers in Caffe. arXiv, 2017.

ResNet-56 for CIFAR-10 Now Supported!

In our latest release, version 0.9.2.30, we now support the ResNet-56 model trained on CIFAR-10 as described by [1], and do so with the newly released CUDA 9.2/cuDNN 7.1.4.

ResNet-56 Model for CIFAR-10

To try this out yourself, just follow the easy steps in the new ResNet tutorial!

New Features
  • New CUDA 9.2 support (requires driver 397.93 or above)
  • New cuDNN 7.1.4 support.
  • Easily switch between CUDA 9.2 and CUDA 9.1.
  • New ResNet-56 support with new model and solver templates.
  • Added ability to print models to image using Save As and selecting the Model image files (*.png) type.
  • Easily apply the settings of one node to all others like it in the Model Editor by right clicking the node and selecting the Apply Settings menu item.
Bug Fixes
  • New bug fixes in the BATCHNORM layer.
  • Solvers now correctly support blobs with no diff.
  • Previous stability issues now appear resolved (by NVIDIA) in latest NVIDIA 397.93 driver.  We now recommend using the 397.93 driver or later.

Happy ‘deep’ learning!

References

[1] Yihui He, ResNet 20 32 44 56 110 for CIFAR10 with caffe. GitHub, 2016.

BATCHNORM and ELU now support cuDNN!

In our latest release, version 0.9.1.86, we have added cuDNN support to both the BATCHNORM and ELU layers which provide speed improvements.  In addition we have added the following new features and fixes.

New Features
  • BATCHNORM layer now supports the cuDNN engine.
  • ELU layer now supports the cuDNN engine.
  • Layer debugging added that allows for easy NAN/INF detection on each pass.
  • Warnings are now supported in the Output Window.
  • Mouse-wheel scrolling has been added to the Toolbox Window.
  • The Model Editor now supports single-stepping both forward and backward passes.
  • The Model Editor now supports drag-n-drop replacement of Neuron layers.
New Bug Fixes
  • BATCHNORM layer (CUDA version) had numerous bug fixes.
  • ACCURACY layer had bug fixes related to NAN output.
  • POOLING configuration dialog now allows global pooling and kernel sizes of zero.
  • Stream synchronization improvements added throughout MyCaffe.
  • T-SNE now properly uses the T-SNE % of images.
  • Bug fixes added to resolve weight importing issues.
Known Issues

There appear to be several stability issues in the NVIDIA driver version 397.31 (as noted here).  We have also experience similar stability issues with this driver and version 397.64 when running on Windows 10.  For these reasons, we recommend using the NVIDIA Driver version 391.35 at this time.

Happy ‘deep’ learning!

Domain-Adversarial Neural Network support added to the SignalPop AI Designer!

Our latest SignalPop AI Designer release, version 0.9.1.70, now supports Domain-Adversarial Neural Networks (DANN) as described by [1].

By adding image overlay support to an updated MNIST Dataset Creator, you can now create both source and target datasets.

Source MNIST Dataset
Target MNIST Dataset

Using the new source and target dataset support you can now easily create DANN networks that use both.

An updated visual editor also supports multiple source and target datasets as shown below with the full DANN model.

DANN Model

The newly added GRADIENTSCALER layer allows for easy gradient reversal which is then added to the bottleneck layer, shown above, to create an adversarial relationship between the two networks.

Try creating a DANN yourself with the easy step-by-step Tutorials that show you how to get up and running with the latest version of the SignalPop AI Designer.

New General Features
  • Projects now optionally support both source and target datasets.
  • A new GRADIENTSCALLER layer has been added for gradient reversals.
  • Full DANN solver and model templates have been added.
  • The MNIST Dataset Creator can now create datasets with an image overlay.
  • We now support the recently released NVIDIA cuDNN 7.1.3.
New Debugging Features
  • We have added single stepping support for both training and testing.
  • A new blob data debugger shows the contents of each blob passing between layers.
  • The model editor has been improved to show models viewable by phase (TRAIN, TEST and RUN).

Happy ‘deep’ learning!

References

[1] Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., . . . Lempitsky, V. (2016). Domain-Adversarial Training of Neural Networks. Journal of Machine Learning Research 17, 1-35.

New beta release 0.9.1.31, synced up with Caffe through 3/26/2018 with cuDNN 7.1.2 support!

Today, we posted a new beta release (0.9.1.31) that is synced up with the native Caffe project through 3/26/2018 and supports the newly released cuDNN 7.1.2.

New Features

1.) Synced up with native Caffe through 3/26/2018 with the following highlights:

  • Added new Swish Layer.
  • Added minor changes and error checking.
  • At this time we are evaluating the new fine tuning changes, but have not added them just yet.

2.) We now support the latest cuDNN 7.1.2 library.

3.) The image evaluator now allows you to choose between the CAFFE and CUDNN engines for deconvolution.

Happy ‘deep’ learning!