MyCaffe  1.12.2.41
Deep learning software for Windows C# programmers.
MyCaffe.trainers.MyCaffeTrainerRL Class Reference

(Depreciated - use MyCaffeTrainerDual instead.) The MyCaffeTraininerRL is used to perform reinforcement learning training tasks on an instance of the MyCaffeControl. More...

Inheritance diagram for MyCaffe.trainers.MyCaffeTrainerRL:
MyCaffe.trainers.IXMyCaffeCustomTrainerRL MyCaffe.trainers.IxTrainerCallback MyCaffe.trainers.IXMyCaffeCustomTrainer

Public Member Functions

 MyCaffeTrainerRL ()
 The constructor. More...
 
 MyCaffeTrainerRL (IContainer container)
 The constructor. More...
 
bool GetUpdateSnapshot (out int nIteration, out double dfAccuracy)
 Returns true when the training is ready for a snap-shot, false otherwise. More...
 
DatasetDescriptor GetDatasetOverride (int nProjectID, ConnectInfo ci=null)
 Returns a dataset override to use (if any) instead of the project's dataset. If there is no dataset override null is returned and the project's dataset is used. More...
 
void CleanUp ()
 Releases any resources used by the component. More...
 
void Initialize (string strProperties, IXMyCaffeCustomTrainerCallback icallback)
 Initializes a new custom trainer by loading the key-value pair of properties into the property set. More...
 
ResultCollection RunOne (Component mycaffe, int nDelay=1000)
 Create a new trainer and use it to run a single run cycle. More...
 
byte[] Run (Component mycaffe, int nN, out string type)
 Run the network using the run technique implemented by this trainer. More...
 
void Test (Component mycaffe, int nIterationOverride, ITERATOR_TYPE type=ITERATOR_TYPE.ITERATION)
 Create a new trainer and use it to run a test cycle. More...
 
void Train (Component mycaffe, int nIterationOverride, ITERATOR_TYPE type=ITERATOR_TYPE.ITERATION, TRAIN_STEP step=TRAIN_STEP.NONE)
 Create a new trainer and use it to run a training cycle. More...
 
void OnInitialize (InitializeArgs e)
 The OnIntialize callback fires when initializing the trainer. More...
 
void OnShutdown ()
 The OnShutdown callback fires when shutting down the trainer. More...
 
void OnGetData (GetDataArgs e)
 The OnGetData callback fires from within the Train method and is used to get a new observation data. More...
 
void OnUpdateStatus (GetStatusArgs e)
 The OnGetStatus callback fires on each iteration within the Train method. More...
 
void OnWait (WaitArgs e)
 The OnWait callback fires when waiting for a shutdown. More...
 
double GetProperty (string strProp)
 Return a property value from the trainer. More...
 
void OpenUi ()
 Open the user interface for the trainer, of one exists. More...
 

Protected Member Functions

virtual DatasetDescriptor get_dataset_override (int nProjectID, ConnectInfo ci=null)
 Returns a dataset override to use (if any) instead of the project's dataset. If there is no dataset override null is returned and the project's dataset is used. More...
 
virtual string get_information ()
 Returns information describing the specific trainer, such as the gym used, if any. More...
 
virtual IxTrainerRL create_trainerD (Component caffe)
 Optionally overridden to return a new type of trainer. More...
 
virtual IxTrainerRL create_trainerF (Component caffe)
 Optionally overridden to return a new type of trainer. More...
 
virtual void dispose ()
 Override to dispose of resources used. More...
 
virtual void initialize (InitializeArgs e)
 Override called by the Initialize method of the trainer. More...
 
virtual void shutdown ()
 Override called from within the CleanUp method. More...
 
virtual bool getData (GetDataArgs e)
 Override called by the OnGetData event fired by the Trainer to retrieve a new set of observation collections making up a set of experiences. More...
 
virtual bool get_update_snapshot (out int nIteration, out double dfAccuracy)
 Returns true when the training is ready for a snap-shot, false otherwise. More...
 
virtual void openUi ()
 Called by OpenUi, override this when a UI (via WCF) should be displayed. More...
 
override void Dispose (bool disposing)
 Clean up any resources being used. More...
 

Protected Attributes

CryptoRandom m_random = new CryptoRandom()
 Random number generator used to get initial actions, etc. More...
 
PropertySet m_properties = null
 Specifies the properties parsed from the key-value pair passed to the Initialize method. More...
 
int m_nProjectID = 0
 Specifies the project ID of the project held by the instance of MyCaffe. More...
 
ConnectInfo m_dsCi = null
 Optionally, specifies the dataset connection info, or null. More...
 

Properties

virtual string name [get]
 Overriden to give the actual name of the custom trainer. More...
 
virtual TRAINING_CATEGORY category [get]
 Override when using a training method other than the REINFORCEMENT method (the default). More...
 
Stage Stage [get]
 Returns the Stage.RL type. More...
 
string Name [get]
 Returns the name of the custom trainer. This method calls the 'name' override. More...
 
TRAINING_CATEGORY TrainingCategory [get]
 Returns the training category of the custom trainer (default = REINFORCEMENT). More...
 
bool IsTrainingSupported [get]
 Returns whether or not Training is supported. More...
 
bool IsTestingSupported [get]
 Returns whether or not Testing is supported. More...
 
bool IsRunningSupported [get]
 Returns whether or not Running is supported. More...
 
double? GlobalRewards [get]
 Returns the global rewards based on the reward type specified by the 'RewardType' property. More...
 
double GlobalLoss [get]
 Return the global loss. More...
 
int GlobalEpisodeCount [get]
 Returns the global episode count. More...
 
int GlobalEpisodeMax [get]
 Returns the maximum global episode count. More...
 
double ExplorationRate [get]
 Returns the current exploration rate. More...
 
double OptimalSelectionRate [get]
 Returns the rate of selection from the optimal set with the highest reward (this setting is optional, default = 0). More...
 
string Information [get]
 Returns information describing the trainer. More...
 
- Properties inherited from MyCaffe.trainers.IXMyCaffeCustomTrainer
Stage Stage [get]
 Returns the stage that the trainer is running under based on the trainer type. More...
 
string Name [get]
 Returns the name of the custom trainer. More...
 
TRAINING_CATEGORY TrainingCategory [get]
 Returns the training category supported by the implementer of the interface. More...
 
bool IsTrainingSupported [get]
 Returns true when the 'Train' method is supported - this should almost always be true. More...
 
bool IsTestingSupported [get]
 Returns true when the 'Test' method is supported. More...
 
bool IsRunningSupported [get]
 Returns true when the 'Run' method is supported. More...
 
string Information [get]
 Returns general information about the custom trainer. More...
 

Detailed Description

(Depreciated - use MyCaffeTrainerDual instead.) The MyCaffeTraininerRL is used to perform reinforcement learning training tasks on an instance of the MyCaffeControl.

Currently, the MyCaffeTrainerRL supports the following trainers, each of which are selected with the 'TrainerType=type' property value within the property set specified when calling the Initialize method.

TrainerType=PG.SIMPLE - creates the initial simple policy gradient trainer that only supports single-threaded Sigmoid based models. TrainerType=PG.ST - creates a single-threaded policy gradient trainer that supports both Sigmoid and Softmax based models. TrainerType=PG.MT - creates a multi-threaded policy gradient trainer that supports both Sigmoid and Softmax based models and can train across GPU's.

Other intitialization properties include:

RewardType=VAL - reports the actual reward values. RewardType=MAX - reports the maximum reward value observed (recommended setting)

Threads=# - specifies the number of threads.

GPUs=#,#,... - specifies the GPU's on which to run each thread. The GPU IDs may be the same as the open project or other GPU's in the system. GPU selection starts with the first GPUID in the list, continues to the end, and then wraps back around to the start of the list. For example if you specifiy to use 3 thread with GPUIDs=0,1 the GPUs will be assigned to each thread as follows: Thread0 => GPUID0, Thread1 => GPUID1, Thread2 => GPUID0

Gamma - specifies the discount rate (default = 0.99) UseRawInput - when true the actual input is used directly, otherwise a difference between the current and previous input is used (default = false).

The following settings are used from the Model and Solver descriptions:

Solver: base_lr - specifies the learning rate used. Model: batch_size - specifies how often accumulated gradients are applied.

Definition at line 47 of file MyCaffeTrainerRL.cs.

Constructor & Destructor Documentation

◆ MyCaffeTrainerRL() [1/2]

MyCaffe.trainers.MyCaffeTrainerRL.MyCaffeTrainerRL ( )

The constructor.

Definition at line 100 of file MyCaffeTrainerRL.cs.

◆ MyCaffeTrainerRL() [2/2]

MyCaffe.trainers.MyCaffeTrainerRL.MyCaffeTrainerRL ( IContainer  container)

The constructor.

Parameters
containerThe container of the component.

Definition at line 109 of file MyCaffeTrainerRL.cs.

Member Function Documentation

◆ CleanUp()

void MyCaffe.trainers.MyCaffeTrainerRL.CleanUp ( )

Releases any resources used by the component.

Implements MyCaffe.trainers.IXMyCaffeCustomTrainer.

Definition at line 357 of file MyCaffeTrainerRL.cs.

◆ create_trainerD()

virtual IxTrainerRL MyCaffe.trainers.MyCaffeTrainerRL.create_trainerD ( Component  caffe)
protectedvirtual

Optionally overridden to return a new type of trainer.

Override this method when using the MyCaffeControl that uses the double base type.

Parameters
caffeSpecifies the MyCaffeControl used.
Returns
The IxTraininer interface implemented by the new trainer is returned.

Definition at line 162 of file MyCaffeTrainerRL.cs.

◆ create_trainerF()

virtual IxTrainerRL MyCaffe.trainers.MyCaffeTrainerRL.create_trainerF ( Component  caffe)
protectedvirtual

Optionally overridden to return a new type of trainer.

Override this method when using the MyCaffeControl that uses the double base type.

Parameters
caffeSpecifies the MyCaffeControl used.
Returns
The IxTraininer interface implemented by the new trainer is returned.

Definition at line 195 of file MyCaffeTrainerRL.cs.

◆ dispose()

virtual void MyCaffe.trainers.MyCaffeTrainerRL.dispose ( )
protectedvirtual

Override to dispose of resources used.

Definition at line 223 of file MyCaffeTrainerRL.cs.

◆ Dispose()

override void MyCaffe.trainers.MyCaffeTrainerRL.Dispose ( bool  disposing)
protected

Clean up any resources being used.

Parameters
disposingtrue if managed resources should be disposed; otherwise, false.

Definition at line 14 of file MyCaffeTrainerRL.Designer.cs.

◆ get_dataset_override()

virtual DatasetDescriptor MyCaffe.trainers.MyCaffeTrainerRL.get_dataset_override ( int  nProjectID,
ConnectInfo  ci = null 
)
protectedvirtual

Returns a dataset override to use (if any) instead of the project's dataset. If there is no dataset override null is returned and the project's dataset is used.

Parameters
nProjectIDSpecifies the project ID associated with the trainer (if any)
ciOptionally, specifies the database connection information (default = null).

Definition at line 140 of file MyCaffeTrainerRL.cs.

◆ get_information()

virtual string MyCaffe.trainers.MyCaffeTrainerRL.get_information ( )
protectedvirtual

Returns information describing the specific trainer, such as the gym used, if any.

Returns
The string describing the trainer is returned.

Definition at line 149 of file MyCaffeTrainerRL.cs.

◆ get_update_snapshot()

virtual bool MyCaffe.trainers.MyCaffeTrainerRL.get_update_snapshot ( out int  nIteration,
out double  dfAccuracy 
)
protectedvirtual

Returns true when the training is ready for a snap-shot, false otherwise.

Parameters
nIterationSpecifies the current iteration.
dfAccuracySpecifies the current rewards.

Definition at line 260 of file MyCaffeTrainerRL.cs.

◆ getData()

virtual bool MyCaffe.trainers.MyCaffeTrainerRL.getData ( GetDataArgs  e)
protectedvirtual

Override called by the OnGetData event fired by the Trainer to retrieve a new set of observation collections making up a set of experiences.

Parameters
eSpecifies the getData argments used to return the new observations.
Returns
A value of true is returned when data is retrieved.

Definition at line 250 of file MyCaffeTrainerRL.cs.

◆ GetDatasetOverride()

DatasetDescriptor MyCaffe.trainers.MyCaffeTrainerRL.GetDatasetOverride ( int  nProjectID,
ConnectInfo  ci = null 
)

Returns a dataset override to use (if any) instead of the project's dataset. If there is no dataset override null is returned and the project's dataset is used.

Parameters
nProjectIDSpecifies the project ID associated with the trainer (if any)
ciOptionally, specifies the database connection information (default = null).

Implements MyCaffe.trainers.IXMyCaffeCustomTrainer.

Definition at line 325 of file MyCaffeTrainerRL.cs.

◆ GetProperty()

double MyCaffe.trainers.MyCaffeTrainerRL.GetProperty ( string  strProp)

Return a property value from the trainer.

Parameters
strPropSpecifies the name of property to retrieve.
Returns
The property value is returned.

Implements MyCaffe.trainers.IXMyCaffeCustomTrainer.

Definition at line 588 of file MyCaffeTrainerRL.cs.

◆ GetUpdateSnapshot()

bool MyCaffe.trainers.MyCaffeTrainerRL.GetUpdateSnapshot ( out int  nIteration,
out double  dfAccuracy 
)

Returns true when the training is ready for a snap-shot, false otherwise.

Parameters
nIterationSpecifies the current iteration.
dfAccuracySpecifies the current rewards.

Implements MyCaffe.trainers.IXMyCaffeCustomTrainer.

Definition at line 314 of file MyCaffeTrainerRL.cs.

◆ initialize()

virtual void MyCaffe.trainers.MyCaffeTrainerRL.initialize ( InitializeArgs  e)
protectedvirtual

Override called by the Initialize method of the trainer.

When providing a new trainer, this method is not used.

Parameters
eSpecifies the initialization arguments.

Definition at line 234 of file MyCaffeTrainerRL.cs.

◆ Initialize()

void MyCaffe.trainers.MyCaffeTrainerRL.Initialize ( string  strProperties,
IXMyCaffeCustomTrainerCallback  icallback 
)

Initializes a new custom trainer by loading the key-value pair of properties into the property set.

Parameters
strPropertiesSpecifies the key-value pair of properties each separated by ';'. For example the expected format is 'key1'='value1';'key2'='value2';...
icallbackSpecifies the parent callback.

Implements MyCaffe.trainers.IXMyCaffeCustomTrainer.

Definition at line 383 of file MyCaffeTrainerRL.cs.

◆ OnGetData()

void MyCaffe.trainers.MyCaffeTrainerRL.OnGetData ( GetDataArgs  e)

The OnGetData callback fires from within the Train method and is used to get a new observation data.

Implements MyCaffe.trainers.IxTrainerCallback.

Definition at line 534 of file MyCaffeTrainerRL.cs.

◆ OnInitialize()

void MyCaffe.trainers.MyCaffeTrainerRL.OnInitialize ( InitializeArgs  e)

The OnIntialize callback fires when initializing the trainer.

Implements MyCaffe.trainers.IxTrainerCallback.

Definition at line 518 of file MyCaffeTrainerRL.cs.

◆ OnShutdown()

void MyCaffe.trainers.MyCaffeTrainerRL.OnShutdown ( )

The OnShutdown callback fires when shutting down the trainer.

Implements MyCaffe.trainers.IxTrainerCallback.

Definition at line 526 of file MyCaffeTrainerRL.cs.

◆ OnUpdateStatus()

void MyCaffe.trainers.MyCaffeTrainerRL.OnUpdateStatus ( GetStatusArgs  e)

The OnGetStatus callback fires on each iteration within the Train method.

Implements MyCaffe.trainers.IxTrainerCallback.

Definition at line 542 of file MyCaffeTrainerRL.cs.

◆ OnWait()

void MyCaffe.trainers.MyCaffeTrainerRL.OnWait ( WaitArgs  e)

The OnWait callback fires when waiting for a shutdown.

Implements MyCaffe.trainers.IxTrainerCallback.

Definition at line 578 of file MyCaffeTrainerRL.cs.

◆ openUi()

virtual void MyCaffe.trainers.MyCaffeTrainerRL.openUi ( )
protectedvirtual

Called by OpenUi, override this when a UI (via WCF) should be displayed.

Definition at line 277 of file MyCaffeTrainerRL.cs.

◆ OpenUi()

void MyCaffe.trainers.MyCaffeTrainerRL.OpenUi ( )

Open the user interface for the trainer, of one exists.

Implements MyCaffe.trainers.IXMyCaffeCustomTrainer.

Definition at line 687 of file MyCaffeTrainerRL.cs.

◆ Run()

byte[] MyCaffe.trainers.MyCaffeTrainerRL.Run ( Component  mycaffe,
int  nN,
out string  type 
)

Run the network using the run technique implemented by this trainer.

Parameters
mycaffeSpecifies an instance to the MyCaffeControl component.
nNSpecifies the number of samples to run.
typeSpecifies the output data type returned as a raw byte stream.
Returns
The run results are returned in the same native type as that of the CustomQuery used.

Implements MyCaffe.trainers.IXMyCaffeCustomTrainerRL.

Definition at line 460 of file MyCaffeTrainerRL.cs.

◆ RunOne()

ResultCollection MyCaffe.trainers.MyCaffeTrainerRL.RunOne ( Component  mycaffe,
int  nDelay = 1000 
)

Create a new trainer and use it to run a single run cycle.

Parameters
mycaffeSpecifies the MyCaffeControl to use.
nDelaySpecifies a delay to wait before getting the action.
Returns
The results of the run are returned.

Implements MyCaffe.trainers.IXMyCaffeCustomTrainerRL.

Definition at line 442 of file MyCaffeTrainerRL.cs.

◆ shutdown()

virtual void MyCaffe.trainers.MyCaffeTrainerRL.shutdown ( )
protectedvirtual

Override called from within the CleanUp method.

Definition at line 241 of file MyCaffeTrainerRL.cs.

◆ Test()

void MyCaffe.trainers.MyCaffeTrainerRL.Test ( Component  mycaffe,
int  nIterationOverride,
ITERATOR_TYPE  type = ITERATOR_TYPE.ITERATION 
)

Create a new trainer and use it to run a test cycle.

Parameters
mycaffeSpecifies the MyCaffeControl to use.
nIterationOverrideSpecifies the iterations to run if greater than zero.
typeSpecifies the type of iterator to use.

Implements MyCaffe.trainers.IXMyCaffeCustomTrainer.

Definition at line 482 of file MyCaffeTrainerRL.cs.

◆ Train()

void MyCaffe.trainers.MyCaffeTrainerRL.Train ( Component  mycaffe,
int  nIterationOverride,
ITERATOR_TYPE  type = ITERATOR_TYPE.ITERATION,
TRAIN_STEP  step = TRAIN_STEP.NONE 
)

Create a new trainer and use it to run a training cycle.

Parameters
mycaffeSpecifies the MyCaffeControl to use.
nIterationOverrideSpecifies the iterations to run if greater than zero.
typeSpecifies the type of iterator to use.
stepOptionally, specifies whether or not to step the training for debugging (default = NONE).

Implements MyCaffe.trainers.IXMyCaffeCustomTrainer.

Definition at line 501 of file MyCaffeTrainerRL.cs.

Member Data Documentation

◆ m_dsCi

ConnectInfo MyCaffe.trainers.MyCaffeTrainerRL.m_dsCi = null
protected

Optionally, specifies the dataset connection info, or null.

Definition at line 64 of file MyCaffeTrainerRL.cs.

◆ m_nProjectID

int MyCaffe.trainers.MyCaffeTrainerRL.m_nProjectID = 0
protected

Specifies the project ID of the project held by the instance of MyCaffe.

Definition at line 60 of file MyCaffeTrainerRL.cs.

◆ m_properties

PropertySet MyCaffe.trainers.MyCaffeTrainerRL.m_properties = null
protected

Specifies the properties parsed from the key-value pair passed to the Initialize method.

Definition at line 56 of file MyCaffeTrainerRL.cs.

◆ m_random

CryptoRandom MyCaffe.trainers.MyCaffeTrainerRL.m_random = new CryptoRandom()
protected

Random number generator used to get initial actions, etc.

Definition at line 52 of file MyCaffeTrainerRL.cs.

Property Documentation

◆ category

virtual TRAINING_CATEGORY MyCaffe.trainers.MyCaffeTrainerRL.category
getprotected

Override when using a training method other than the REINFORCEMENT method (the default).

Definition at line 129 of file MyCaffeTrainerRL.cs.

◆ ExplorationRate

double MyCaffe.trainers.MyCaffeTrainerRL.ExplorationRate
get

Returns the current exploration rate.

Definition at line 663 of file MyCaffeTrainerRL.cs.

◆ GlobalEpisodeCount

int MyCaffe.trainers.MyCaffeTrainerRL.GlobalEpisodeCount
get

Returns the global episode count.

Definition at line 647 of file MyCaffeTrainerRL.cs.

◆ GlobalEpisodeMax

int MyCaffe.trainers.MyCaffeTrainerRL.GlobalEpisodeMax
get

Returns the maximum global episode count.

Definition at line 655 of file MyCaffeTrainerRL.cs.

◆ GlobalLoss

double MyCaffe.trainers.MyCaffeTrainerRL.GlobalLoss
get

Return the global loss.

Definition at line 639 of file MyCaffeTrainerRL.cs.

◆ GlobalRewards

double? MyCaffe.trainers.MyCaffeTrainerRL.GlobalRewards
get

Returns the global rewards based on the reward type specified by the 'RewardType' property.

The reward type can be one of the following: 'VAL' - report the global reward value. 'AVE' - report the global reward averaged over all threads. 'MAX' - report maximum global rewards (default)

Definition at line 618 of file MyCaffeTrainerRL.cs.

◆ Information

string MyCaffe.trainers.MyCaffeTrainerRL.Information
get

Returns information describing the trainer.

Definition at line 679 of file MyCaffeTrainerRL.cs.

◆ IsRunningSupported

bool MyCaffe.trainers.MyCaffeTrainerRL.IsRunningSupported
get

Returns whether or not Running is supported.

Definition at line 349 of file MyCaffeTrainerRL.cs.

◆ IsTestingSupported

bool MyCaffe.trainers.MyCaffeTrainerRL.IsTestingSupported
get

Returns whether or not Testing is supported.

Definition at line 341 of file MyCaffeTrainerRL.cs.

◆ IsTrainingSupported

bool MyCaffe.trainers.MyCaffeTrainerRL.IsTrainingSupported
get

Returns whether or not Training is supported.

Definition at line 333 of file MyCaffeTrainerRL.cs.

◆ name

virtual string MyCaffe.trainers.MyCaffeTrainerRL.name
getprotected

Overriden to give the actual name of the custom trainer.

Definition at line 121 of file MyCaffeTrainerRL.cs.

◆ Name

string MyCaffe.trainers.MyCaffeTrainerRL.Name
get

Returns the name of the custom trainer. This method calls the 'name' override.

Definition at line 296 of file MyCaffeTrainerRL.cs.

◆ OptimalSelectionRate

double MyCaffe.trainers.MyCaffeTrainerRL.OptimalSelectionRate
get

Returns the rate of selection from the optimal set with the highest reward (this setting is optional, default = 0).

Definition at line 671 of file MyCaffeTrainerRL.cs.

◆ Stage

Stage MyCaffe.trainers.MyCaffeTrainerRL.Stage
get

Returns the Stage.RL type.

Definition at line 288 of file MyCaffeTrainerRL.cs.

◆ TrainingCategory

TRAINING_CATEGORY MyCaffe.trainers.MyCaffeTrainerRL.TrainingCategory
get

Returns the training category of the custom trainer (default = REINFORCEMENT).

Definition at line 304 of file MyCaffeTrainerRL.cs.


The documentation for this class was generated from the following files: