MyCaffe  1.11.8.27
Deep learning software for Windows C# programmers.
NesterovSolver.cs
1using System;
2using System.Collections.Generic;
3using System.Linq;
4using System.Text;
5using System.Threading;
6using System.IO;
7using MyCaffe.basecode;
8using MyCaffe.db.image;
9using MyCaffe.common;
10using MyCaffe.param;
11
12namespace MyCaffe.solvers
13{
22 public class NesterovSolver<T> : SGDSolver<T>
23 {
40 public NesterovSolver(CudaDnn<T> cuda, Log log, SolverParameter p, CancelEvent evtCancel, AutoResetEvent evtForceSnapshot, AutoResetEvent evtForceTest, IXImageDatabaseBase imgDb, IXPersist<T> persist, int nSolverCount = 1, int nSolverRank = 0, Net<T> shareNet = null, onGetWorkspace getws = null, onSetWorkspace setws = null)
41 : base(cuda, log, p, evtCancel, evtForceSnapshot, evtForceTest, imgDb, persist, nSolverCount, nSolverRank, shareNet, getws, setws)
42 {
43 }
44
51 public override void ComputeUpdateValue(int param_id, double dfRate, int nIterationOverride = -1)
52 {
53 BlobCollection<T> colNetParams = m_net.learnable_parameters;
54
55 if (!colNetParams[param_id].DiffExists)
56 return;
57
58 List<double?> net_params_lr = m_net.params_lr;
59 T fMomentum = Utility.ConvertVal<T>(m_param.momentum);
60 T fLocalRate = Utility.ConvertVal<T>(dfRate * net_params_lr[param_id].GetValueOrDefault(0));
61
62 // Compute the update to history, then copy it to the parameter diff.
63 m_cuda.nesterov_update(colNetParams[param_id].count(), colNetParams[param_id].mutable_gpu_diff, m_colHistory[param_id].mutable_gpu_data, fMomentum, fLocalRate);
64 }
65 }
66}
The CancelEvent provides an extension to the manual cancel event that allows for overriding the manua...
Definition: CancelEvent.cs:17
The Log class provides general output in text form.
Definition: Log.cs:13
The Utility class provides general utility funtions.
Definition: Utility.cs:35
The BlobCollection contains a list of Blobs.
The CudaDnn object is the main interface to the Low-Level Cuda C++ DLL.
Definition: CudaDnn.cs:851
Connects Layer's together into a direct acrylic graph (DAG) specified by a NetParameter
Definition: Net.cs:23
The SolverParameter is a parameter for the solver, specifying the train and test networks.
double momentum
Specifies the momentum value - used by all solvers EXCEPT the 'AdaGrad' and 'RMSProp' solvers....
Use Nesterov's accelerated gradient Solver, which is similar to SGD, but the error gradient is comput...
NesterovSolver(CudaDnn< T > cuda, Log log, SolverParameter p, CancelEvent evtCancel, AutoResetEvent evtForceSnapshot, AutoResetEvent evtForceTest, IXImageDatabaseBase imgDb, IXPersist< T > persist, int nSolverCount=1, int nSolverRank=0, Net< T > shareNet=null, onGetWorkspace getws=null, onSetWorkspace setws=null)
The NesterovSolver constructor.
override void ComputeUpdateValue(int param_id, double dfRate, int nIterationOverride=-1)
Compute the Nesterov update value that will be applied to a learnable blobs in the training Net.
Stochastic Gradient Descent solver with momentum updates weights by a linear combination of the negat...
Definition: SGDSolver.cs:22
BlobCollection< T > m_colHistory
History maintains the historical momentum data.
Definition: SGDSolver.cs:26
SolverParameter m_param
Specifies the SolverParameter that defines how the Solver operates.
Definition: Solver.cs:40
CudaDnn< T > m_cuda
Specifies the instance of CudaDnn used by the Solver that provides a connection to Cuda.
Definition: Solver.cs:32
Net< T > m_net
Specifies the training Net.
Definition: Solver.cs:44
The IXImageDatabaseBase interface defines the general interface to the in-memory image database.
Definition: Interfaces.cs:415
The IXPersist interface is used by the CaffeControl to load and save weights.
Definition: Interfaces.cs:175
The MyCaffe.basecode contains all generic types used throughout MyCaffe.
Definition: Annotation.cs:12
The MyCaffe.common namespace contains common MyCaffe classes.
Definition: BatchInput.cs:8
The MyCaffe.db.image namespace contains all image database related classes.
Definition: Database.cs:18
The MyCaffe.param namespace contains parameters used to create models.
The MyCaffe.solvers namespace contains all solver classes, including the base Solver.
The MyCaffe namespace contains the main body of MyCaffe code that closesly tracks the C++ Caffe open-...
Definition: Annotation.cs:12