MyCaffe
1.12.2.41
Deep learning software for Windows C# programmers.
|
(Depreciated - use MyCaffeTrainerDual instead.) The MyCaffeTraininerRL is used to perform reinforcement learning training tasks on an instance of the MyCaffeControl. More...
Public Member Functions | |
MyCaffeTrainerRL () | |
The constructor. More... | |
MyCaffeTrainerRL (IContainer container) | |
The constructor. More... | |
bool | GetUpdateSnapshot (out int nIteration, out double dfAccuracy) |
Returns true when the training is ready for a snap-shot, false otherwise. More... | |
DatasetDescriptor | GetDatasetOverride (int nProjectID, ConnectInfo ci=null) |
Returns a dataset override to use (if any) instead of the project's dataset. If there is no dataset override null is returned and the project's dataset is used. More... | |
void | CleanUp () |
Releases any resources used by the component. More... | |
void | Initialize (string strProperties, IXMyCaffeCustomTrainerCallback icallback) |
Initializes a new custom trainer by loading the key-value pair of properties into the property set. More... | |
ResultCollection | RunOne (Component mycaffe, int nDelay=1000) |
Create a new trainer and use it to run a single run cycle. More... | |
byte[] | Run (Component mycaffe, int nN, out string type) |
Run the network using the run technique implemented by this trainer. More... | |
void | Test (Component mycaffe, int nIterationOverride, ITERATOR_TYPE type=ITERATOR_TYPE.ITERATION) |
Create a new trainer and use it to run a test cycle. More... | |
void | Train (Component mycaffe, int nIterationOverride, ITERATOR_TYPE type=ITERATOR_TYPE.ITERATION, TRAIN_STEP step=TRAIN_STEP.NONE) |
Create a new trainer and use it to run a training cycle. More... | |
void | OnInitialize (InitializeArgs e) |
The OnIntialize callback fires when initializing the trainer. More... | |
void | OnShutdown () |
The OnShutdown callback fires when shutting down the trainer. More... | |
void | OnGetData (GetDataArgs e) |
The OnGetData callback fires from within the Train method and is used to get a new observation data. More... | |
void | OnUpdateStatus (GetStatusArgs e) |
The OnGetStatus callback fires on each iteration within the Train method. More... | |
void | OnWait (WaitArgs e) |
The OnWait callback fires when waiting for a shutdown. More... | |
double | GetProperty (string strProp) |
Return a property value from the trainer. More... | |
void | OpenUi () |
Open the user interface for the trainer, of one exists. More... | |
Protected Member Functions | |
virtual DatasetDescriptor | get_dataset_override (int nProjectID, ConnectInfo ci=null) |
Returns a dataset override to use (if any) instead of the project's dataset. If there is no dataset override null is returned and the project's dataset is used. More... | |
virtual string | get_information () |
Returns information describing the specific trainer, such as the gym used, if any. More... | |
virtual IxTrainerRL | create_trainerD (Component caffe) |
Optionally overridden to return a new type of trainer. More... | |
virtual IxTrainerRL | create_trainerF (Component caffe) |
Optionally overridden to return a new type of trainer. More... | |
virtual void | dispose () |
Override to dispose of resources used. More... | |
virtual void | initialize (InitializeArgs e) |
Override called by the Initialize method of the trainer. More... | |
virtual void | shutdown () |
Override called from within the CleanUp method. More... | |
virtual bool | getData (GetDataArgs e) |
Override called by the OnGetData event fired by the Trainer to retrieve a new set of observation collections making up a set of experiences. More... | |
virtual bool | get_update_snapshot (out int nIteration, out double dfAccuracy) |
Returns true when the training is ready for a snap-shot, false otherwise. More... | |
virtual void | openUi () |
Called by OpenUi, override this when a UI (via WCF) should be displayed. More... | |
override void | Dispose (bool disposing) |
Clean up any resources being used. More... | |
Protected Attributes | |
CryptoRandom | m_random = new CryptoRandom() |
Random number generator used to get initial actions, etc. More... | |
PropertySet | m_properties = null |
Specifies the properties parsed from the key-value pair passed to the Initialize method. More... | |
int | m_nProjectID = 0 |
Specifies the project ID of the project held by the instance of MyCaffe. More... | |
ConnectInfo | m_dsCi = null |
Optionally, specifies the dataset connection info, or null. More... | |
Properties | |
virtual string | name [get] |
Overriden to give the actual name of the custom trainer. More... | |
virtual TRAINING_CATEGORY | category [get] |
Override when using a training method other than the REINFORCEMENT method (the default). More... | |
Stage | Stage [get] |
Returns the Stage.RL type. More... | |
string | Name [get] |
Returns the name of the custom trainer. This method calls the 'name' override. More... | |
TRAINING_CATEGORY | TrainingCategory [get] |
Returns the training category of the custom trainer (default = REINFORCEMENT). More... | |
bool | IsTrainingSupported [get] |
Returns whether or not Training is supported. More... | |
bool | IsTestingSupported [get] |
Returns whether or not Testing is supported. More... | |
bool | IsRunningSupported [get] |
Returns whether or not Running is supported. More... | |
double? | GlobalRewards [get] |
Returns the global rewards based on the reward type specified by the 'RewardType' property. More... | |
double | GlobalLoss [get] |
Return the global loss. More... | |
int | GlobalEpisodeCount [get] |
Returns the global episode count. More... | |
int | GlobalEpisodeMax [get] |
Returns the maximum global episode count. More... | |
double | ExplorationRate [get] |
Returns the current exploration rate. More... | |
double | OptimalSelectionRate [get] |
Returns the rate of selection from the optimal set with the highest reward (this setting is optional, default = 0). More... | |
string | Information [get] |
Returns information describing the trainer. More... | |
Properties inherited from MyCaffe.trainers.IXMyCaffeCustomTrainer | |
Stage | Stage [get] |
Returns the stage that the trainer is running under based on the trainer type. More... | |
string | Name [get] |
Returns the name of the custom trainer. More... | |
TRAINING_CATEGORY | TrainingCategory [get] |
Returns the training category supported by the implementer of the interface. More... | |
bool | IsTrainingSupported [get] |
Returns true when the 'Train' method is supported - this should almost always be true. More... | |
bool | IsTestingSupported [get] |
Returns true when the 'Test' method is supported. More... | |
bool | IsRunningSupported [get] |
Returns true when the 'Run' method is supported. More... | |
string | Information [get] |
Returns general information about the custom trainer. More... | |
(Depreciated - use MyCaffeTrainerDual instead.) The MyCaffeTraininerRL is used to perform reinforcement learning training tasks on an instance of the MyCaffeControl.
Currently, the MyCaffeTrainerRL supports the following trainers, each of which are selected with the 'TrainerType=type' property value within the property set specified when calling the Initialize method.
TrainerType=PG.SIMPLE - creates the initial simple policy gradient trainer that only supports single-threaded Sigmoid based models. TrainerType=PG.ST - creates a single-threaded policy gradient trainer that supports both Sigmoid and Softmax based models. TrainerType=PG.MT - creates a multi-threaded policy gradient trainer that supports both Sigmoid and Softmax based models and can train across GPU's.
Other intitialization properties include:
RewardType=VAL - reports the actual reward values. RewardType=MAX - reports the maximum reward value observed (recommended setting)
Threads=# - specifies the number of threads.
GPUs=#,#,... - specifies the GPU's on which to run each thread. The GPU IDs may be the same as the open project or other GPU's in the system. GPU selection starts with the first GPUID in the list, continues to the end, and then wraps back around to the start of the list. For example if you specifiy to use 3 thread with GPUIDs=0,1 the GPUs will be assigned to each thread as follows: Thread0 => GPUID0, Thread1 => GPUID1, Thread2 => GPUID0
Gamma - specifies the discount rate (default = 0.99) UseRawInput - when true the actual input is used directly, otherwise a difference between the current and previous input is used (default = false).
The following settings are used from the Model and Solver descriptions:
Solver: base_lr - specifies the learning rate used. Model: batch_size - specifies how often accumulated gradients are applied.
Definition at line 47 of file MyCaffeTrainerRL.cs.
MyCaffe.trainers.MyCaffeTrainerRL.MyCaffeTrainerRL | ( | ) |
The constructor.
Definition at line 100 of file MyCaffeTrainerRL.cs.
MyCaffe.trainers.MyCaffeTrainerRL.MyCaffeTrainerRL | ( | IContainer | container | ) |
The constructor.
container | The container of the component. |
Definition at line 109 of file MyCaffeTrainerRL.cs.
void MyCaffe.trainers.MyCaffeTrainerRL.CleanUp | ( | ) |
Releases any resources used by the component.
Implements MyCaffe.trainers.IXMyCaffeCustomTrainer.
Definition at line 357 of file MyCaffeTrainerRL.cs.
|
protectedvirtual |
Optionally overridden to return a new type of trainer.
Override this method when using the MyCaffeControl that uses the double base type.
caffe | Specifies the MyCaffeControl used. |
Definition at line 162 of file MyCaffeTrainerRL.cs.
|
protectedvirtual |
Optionally overridden to return a new type of trainer.
Override this method when using the MyCaffeControl that uses the double base type.
caffe | Specifies the MyCaffeControl used. |
Definition at line 195 of file MyCaffeTrainerRL.cs.
|
protectedvirtual |
Override to dispose of resources used.
Definition at line 223 of file MyCaffeTrainerRL.cs.
|
protected |
Clean up any resources being used.
disposing | true if managed resources should be disposed; otherwise, false. |
Definition at line 14 of file MyCaffeTrainerRL.Designer.cs.
|
protectedvirtual |
Returns a dataset override to use (if any) instead of the project's dataset. If there is no dataset override null is returned and the project's dataset is used.
nProjectID | Specifies the project ID associated with the trainer (if any) |
ci | Optionally, specifies the database connection information (default = null). |
Definition at line 140 of file MyCaffeTrainerRL.cs.
|
protectedvirtual |
Returns information describing the specific trainer, such as the gym used, if any.
Definition at line 149 of file MyCaffeTrainerRL.cs.
|
protectedvirtual |
Returns true when the training is ready for a snap-shot, false otherwise.
nIteration | Specifies the current iteration. |
dfAccuracy | Specifies the current rewards. |
Definition at line 260 of file MyCaffeTrainerRL.cs.
|
protectedvirtual |
Override called by the OnGetData event fired by the Trainer to retrieve a new set of observation collections making up a set of experiences.
e | Specifies the getData argments used to return the new observations. |
Definition at line 250 of file MyCaffeTrainerRL.cs.
DatasetDescriptor MyCaffe.trainers.MyCaffeTrainerRL.GetDatasetOverride | ( | int | nProjectID, |
ConnectInfo | ci = null |
||
) |
Returns a dataset override to use (if any) instead of the project's dataset. If there is no dataset override null is returned and the project's dataset is used.
nProjectID | Specifies the project ID associated with the trainer (if any) |
ci | Optionally, specifies the database connection information (default = null). |
Implements MyCaffe.trainers.IXMyCaffeCustomTrainer.
Definition at line 325 of file MyCaffeTrainerRL.cs.
double MyCaffe.trainers.MyCaffeTrainerRL.GetProperty | ( | string | strProp | ) |
Return a property value from the trainer.
strProp | Specifies the name of property to retrieve. |
Implements MyCaffe.trainers.IXMyCaffeCustomTrainer.
Definition at line 588 of file MyCaffeTrainerRL.cs.
bool MyCaffe.trainers.MyCaffeTrainerRL.GetUpdateSnapshot | ( | out int | nIteration, |
out double | dfAccuracy | ||
) |
Returns true when the training is ready for a snap-shot, false otherwise.
nIteration | Specifies the current iteration. |
dfAccuracy | Specifies the current rewards. |
Implements MyCaffe.trainers.IXMyCaffeCustomTrainer.
Definition at line 314 of file MyCaffeTrainerRL.cs.
|
protectedvirtual |
Override called by the Initialize method of the trainer.
When providing a new trainer, this method is not used.
e | Specifies the initialization arguments. |
Definition at line 234 of file MyCaffeTrainerRL.cs.
void MyCaffe.trainers.MyCaffeTrainerRL.Initialize | ( | string | strProperties, |
IXMyCaffeCustomTrainerCallback | icallback | ||
) |
Initializes a new custom trainer by loading the key-value pair of properties into the property set.
strProperties | Specifies the key-value pair of properties each separated by ';'. For example the expected format is 'key1'='value1';'key2'='value2';... |
icallback | Specifies the parent callback. |
Implements MyCaffe.trainers.IXMyCaffeCustomTrainer.
Definition at line 383 of file MyCaffeTrainerRL.cs.
void MyCaffe.trainers.MyCaffeTrainerRL.OnGetData | ( | GetDataArgs | e | ) |
The OnGetData callback fires from within the Train method and is used to get a new observation data.
Implements MyCaffe.trainers.IxTrainerCallback.
Definition at line 534 of file MyCaffeTrainerRL.cs.
void MyCaffe.trainers.MyCaffeTrainerRL.OnInitialize | ( | InitializeArgs | e | ) |
The OnIntialize callback fires when initializing the trainer.
Implements MyCaffe.trainers.IxTrainerCallback.
Definition at line 518 of file MyCaffeTrainerRL.cs.
void MyCaffe.trainers.MyCaffeTrainerRL.OnShutdown | ( | ) |
The OnShutdown callback fires when shutting down the trainer.
Implements MyCaffe.trainers.IxTrainerCallback.
Definition at line 526 of file MyCaffeTrainerRL.cs.
void MyCaffe.trainers.MyCaffeTrainerRL.OnUpdateStatus | ( | GetStatusArgs | e | ) |
The OnGetStatus callback fires on each iteration within the Train method.
Implements MyCaffe.trainers.IxTrainerCallback.
Definition at line 542 of file MyCaffeTrainerRL.cs.
void MyCaffe.trainers.MyCaffeTrainerRL.OnWait | ( | WaitArgs | e | ) |
The OnWait callback fires when waiting for a shutdown.
Implements MyCaffe.trainers.IxTrainerCallback.
Definition at line 578 of file MyCaffeTrainerRL.cs.
|
protectedvirtual |
Called by OpenUi, override this when a UI (via WCF) should be displayed.
Definition at line 277 of file MyCaffeTrainerRL.cs.
void MyCaffe.trainers.MyCaffeTrainerRL.OpenUi | ( | ) |
Open the user interface for the trainer, of one exists.
Implements MyCaffe.trainers.IXMyCaffeCustomTrainer.
Definition at line 687 of file MyCaffeTrainerRL.cs.
byte[] MyCaffe.trainers.MyCaffeTrainerRL.Run | ( | Component | mycaffe, |
int | nN, | ||
out string | type | ||
) |
Run the network using the run technique implemented by this trainer.
mycaffe | Specifies an instance to the MyCaffeControl component. |
nN | Specifies the number of samples to run. |
type | Specifies the output data type returned as a raw byte stream. |
Implements MyCaffe.trainers.IXMyCaffeCustomTrainerRL.
Definition at line 460 of file MyCaffeTrainerRL.cs.
ResultCollection MyCaffe.trainers.MyCaffeTrainerRL.RunOne | ( | Component | mycaffe, |
int | nDelay = 1000 |
||
) |
Create a new trainer and use it to run a single run cycle.
mycaffe | Specifies the MyCaffeControl to use. |
nDelay | Specifies a delay to wait before getting the action. |
Implements MyCaffe.trainers.IXMyCaffeCustomTrainerRL.
Definition at line 442 of file MyCaffeTrainerRL.cs.
|
protectedvirtual |
Override called from within the CleanUp method.
Definition at line 241 of file MyCaffeTrainerRL.cs.
void MyCaffe.trainers.MyCaffeTrainerRL.Test | ( | Component | mycaffe, |
int | nIterationOverride, | ||
ITERATOR_TYPE | type = ITERATOR_TYPE.ITERATION |
||
) |
Create a new trainer and use it to run a test cycle.
mycaffe | Specifies the MyCaffeControl to use. |
nIterationOverride | Specifies the iterations to run if greater than zero. |
type | Specifies the type of iterator to use. |
Implements MyCaffe.trainers.IXMyCaffeCustomTrainer.
Definition at line 482 of file MyCaffeTrainerRL.cs.
void MyCaffe.trainers.MyCaffeTrainerRL.Train | ( | Component | mycaffe, |
int | nIterationOverride, | ||
ITERATOR_TYPE | type = ITERATOR_TYPE.ITERATION , |
||
TRAIN_STEP | step = TRAIN_STEP.NONE |
||
) |
Create a new trainer and use it to run a training cycle.
mycaffe | Specifies the MyCaffeControl to use. |
nIterationOverride | Specifies the iterations to run if greater than zero. |
type | Specifies the type of iterator to use. |
step | Optionally, specifies whether or not to step the training for debugging (default = NONE). |
Implements MyCaffe.trainers.IXMyCaffeCustomTrainer.
Definition at line 501 of file MyCaffeTrainerRL.cs.
|
protected |
Optionally, specifies the dataset connection info, or null.
Definition at line 64 of file MyCaffeTrainerRL.cs.
|
protected |
Specifies the project ID of the project held by the instance of MyCaffe.
Definition at line 60 of file MyCaffeTrainerRL.cs.
|
protected |
Specifies the properties parsed from the key-value pair passed to the Initialize method.
Definition at line 56 of file MyCaffeTrainerRL.cs.
|
protected |
Random number generator used to get initial actions, etc.
Definition at line 52 of file MyCaffeTrainerRL.cs.
|
getprotected |
Override when using a training method other than the REINFORCEMENT method (the default).
Definition at line 129 of file MyCaffeTrainerRL.cs.
|
get |
Returns the current exploration rate.
Definition at line 663 of file MyCaffeTrainerRL.cs.
|
get |
Returns the global episode count.
Definition at line 647 of file MyCaffeTrainerRL.cs.
|
get |
Returns the maximum global episode count.
Definition at line 655 of file MyCaffeTrainerRL.cs.
|
get |
Return the global loss.
Definition at line 639 of file MyCaffeTrainerRL.cs.
|
get |
Returns the global rewards based on the reward type specified by the 'RewardType' property.
The reward type can be one of the following: 'VAL' - report the global reward value. 'AVE' - report the global reward averaged over all threads. 'MAX' - report maximum global rewards (default)
Definition at line 618 of file MyCaffeTrainerRL.cs.
|
get |
Returns information describing the trainer.
Definition at line 679 of file MyCaffeTrainerRL.cs.
|
get |
Returns whether or not Running is supported.
Definition at line 349 of file MyCaffeTrainerRL.cs.
|
get |
Returns whether or not Testing is supported.
Definition at line 341 of file MyCaffeTrainerRL.cs.
|
get |
Returns whether or not Training is supported.
Definition at line 333 of file MyCaffeTrainerRL.cs.
|
getprotected |
Overriden to give the actual name of the custom trainer.
Definition at line 121 of file MyCaffeTrainerRL.cs.
|
get |
Returns the name of the custom trainer. This method calls the 'name' override.
Definition at line 296 of file MyCaffeTrainerRL.cs.
|
get |
Returns the rate of selection from the optimal set with the highest reward (this setting is optional, default = 0).
Definition at line 671 of file MyCaffeTrainerRL.cs.
|
get |
Returns the Stage.RL type.
Definition at line 288 of file MyCaffeTrainerRL.cs.
|
get |
Returns the training category of the custom trainer (default = REINFORCEMENT).
Definition at line 304 of file MyCaffeTrainerRL.cs.