Specifies the parameters for the LSTMAttentionLayer that provides an attention based LSTM layer used for decoding in an encoder/decoder based model. More...

Inheritance diagram for MyCaffe.param.LSTMAttentionParameter:

Public Member Functions
	LSTMAttentionParameter ()
	Constructor for the parameter. More...

override object	Load (System.IO.BinaryReader br, bool bNewInstance=true)
	Load the parameter from a binary reader. More...

override void	Copy (LayerParameterBase src)
	Copy on parameter to another. More...

override LayerParameterBase	Clone ()
	Creates a new copy of this instance of the parameter. More...

override RawProto	ToProto (string strName)
	Convert the parameter into a RawProto. More...

Public Member Functions inherited from MyCaffe.param.LayerParameterBase
	LayerParameterBase ()
	Constructor for the parameter. More...

virtual string	PrepareRunModelInputs ()
	This method gives derivative classes a chance specify model inputs required by the run model. More...

virtual void	PrepareRunModel (LayerParameter p)
	This method gives derivative classes a chance to prepare the layer for a run-model. More...

void	Save (BinaryWriter bw)
	Save this parameter to a binary writer. More...

abstract object	Load (BinaryReader br, bool bNewInstance=true)
	Load the parameter from a binary reader. More...

Public Member Functions inherited from MyCaffe.basecode.BaseParameter
	BaseParameter ()
	Constructor for the parameter. More...

virtual bool	Compare (BaseParameter p)
	Compare this parameter to another parameter. More...

Static Public Member Functions
static LSTMAttentionParameter	FromProto (RawProto rp)
	Parses the parameter from a RawProto. More...

Static Public Member Functions inherited from MyCaffe.basecode.BaseParameter
static double	ParseDouble (string strVal)
	Parse double values using the US culture if the decimal separator = '.', then using the native culture, and if then lastly trying the US culture to handle prototypes containing '.' as the separator, yet parsed in a culture that does not use '.' as a decimal. More...

static bool	TryParse (string strVal, out double df)
	Parse double values using the US culture if the decimal separator = '.', then using the native culture, and if then lastly trying the US culture to handle prototypes containing '.' as the separator, yet parsed in a culture that does not use '.' as a decimal. More...

static float	ParseFloat (string strVal)
	Parse float values using the US culture if the decimal separator = '.', then using the native culture, and if then lastly trying the US culture to handle prototypes containing '.' as the separator, yet parsed in a culture that does not use '.' as a decimal. More...

static bool	TryParse (string strVal, out float f)
	Parse doufloatble values using the US culture if the decimal separator = '.', then using the native culture, and if then lastly trying the US culture to handle prototypes containing '.' as the separator, yet parsed in a culture that does not use '.' as a decimal. More...

Properties
uint	num_output `[getset]`
	Specifies the number of outputs for the layer. More...

uint	num_output_ip `[getset]`
	Specifies the number of IP outputs for the layer. Note, when 0, no inner product is performed. More...

double	clipping_threshold `[getset]`
	Specifies the gradient clipping threshold, default = 0.0 (i.e. no clipping). More...

FillerParameter	weight_filler `[getset]`
	Specifies the filler parameters for the weight filler. More...

FillerParameter	bias_filler `[getset]`
	Specifies the filler parameters for the bias filler. More...

bool	enable_clockwork_forgetgate_bias `[getset]`
	When enabled, the forget gate bias is set to 5.0. More...

bool	enable_attention `[getset]`
	(default=false) When enabled, attention is applied to the input state on each cycle through the LSTM. Attention is used with encoder/decoder models. When disabled, this layer operates like a standard LSTM layer with input in the shape T,B,I, where T=timesteps, b=batch and i=input. More...

Additional Inherited Members
Public Types inherited from MyCaffe.param.LayerParameterBase
enum	LABEL_TYPE { NONE , SINGLE , MULTIPLE , ONLY_ONE }
	Defines the label type. More...

Detailed Description

Specifies the parameters for the LSTMAttentionLayer that provides an attention based LSTM layer used for decoding in an encoder/decoder based model.

See also: Attention Is All You Need by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin, 2017, arXiv:1706:03762; Attention is All You Need by Jenny Ching, 2019, Medium; [Attention is All You Need in Speech Separation] by Cem Subakan, Mirco Ravanelli, Samuele Cornell, Mirko Bronzi, Jianyuan Zhong, 2020, arXiv:2010.13154; Sequence to Sequence Learning with Neural Networks by Ilya Sutskever, Oriol Vinyals, and Quoc V. Le, 2014, arXiv:1409.3215

The AttentionLayer implementation was inspired by the C# Seq2SeqLearn implementation by mashmawy for language translation,

See also: mashmawy/Seq2SeqLearn distributed under MIT license.

And also inspired by the C# ChatBot implementation by HectorPulido which uses Seq2SeqLearn

See also: HectorPulido/Chatbot-seq2seq-C- distributed under MIT license.

Definition at line 28 of file LSTMAttentionParameter.cs.

Constructor & Destructor Documentation

◆ LSTMAttentionParameter()

MyCaffe.param.LSTMAttentionParameter.LSTMAttentionParameter ( )

Constructor for the parameter.

Definition at line 39 of file LSTMAttentionParameter.cs.

Member Function Documentation

◆ Clone()

override LayerParameterBase MyCaffe.param.LSTMAttentionParameter.Clone ( )

virtual

Creates a new copy of this instance of the parameter.

Returns: A new instance of this parameter is returned.

Implements MyCaffe.param.LayerParameterBase.

Definition at line 146 of file LSTMAttentionParameter.cs.

◆ Copy()

override void MyCaffe.param.LSTMAttentionParameter.Copy ( LayerParameterBase src )

virtual

Copy on parameter to another.

Parameters

src	Specifies the parameter to copy.

Implements MyCaffe.param.LayerParameterBase.

Definition at line 132 of file LSTMAttentionParameter.cs.

◆ FromProto()

static LSTMAttentionParameter MyCaffe.param.LSTMAttentionParameter.FromProto ( RawProto rp )

static

Parses the parameter from a RawProto.

Parameters

rp	Specifies the RawProto to parse.

Returns: A new instance of the parameter is returned.

Definition at line 184 of file LSTMAttentionParameter.cs.

◆ Load()

override object MyCaffe.param.LSTMAttentionParameter.Load	(	System.IO.BinaryReader	br,
		bool	bNewInstance = `true`
	)

Load the parameter from a binary reader.

Parameters

br	Specifies the binary reader.
bNewInstance	When true a new instance is created (the default), otherwise the existing instance is loaded from the binary reader.

Returns: Returns an instance of the parameter.

Definition at line 120 of file LSTMAttentionParameter.cs.

◆ ToProto()

override RawProto MyCaffe.param.LSTMAttentionParameter.ToProto ( string strName )

virtual

Convert the parameter into a RawProto.

Parameters

strName Specifies the name to associate with the RawProto.

Returns: The new RawProto is returned.

Implements MyCaffe.basecode.BaseParameter.

Definition at line 158 of file LSTMAttentionParameter.cs.

Property Documentation

◆ bias_filler

FillerParameter MyCaffe.param.LSTMAttentionParameter.bias_filler

getset

Specifies the filler parameters for the bias filler.

Definition at line 89 of file LSTMAttentionParameter.cs.

◆ clipping_threshold

double MyCaffe.param.LSTMAttentionParameter.clipping_threshold

getset

Specifies the gradient clipping threshold, default = 0.0 (i.e. no clipping).

Definition at line 67 of file LSTMAttentionParameter.cs.

◆ enable_attention

bool MyCaffe.param.LSTMAttentionParameter.enable_attention

getset

(default=false) When enabled, attention is applied to the input state on each cycle through the LSTM. Attention is used with encoder/decoder models. When disabled, this layer operates like a standard LSTM layer with input in the shape T,B,I, where T=timesteps, b=batch and i=input.

Definition at line 113 of file LSTMAttentionParameter.cs.

◆ enable_clockwork_forgetgate_bias

bool MyCaffe.param.LSTMAttentionParameter.enable_clockwork_forgetgate_bias

getset

When enabled, the forget gate bias is set to 5.0.

See also: A Clockwork RNN by Koutnik, et al., 2014

Definition at line 102 of file LSTMAttentionParameter.cs.

◆ num_output

uint MyCaffe.param.LSTMAttentionParameter.num_output

getset

Specifies the number of outputs for the layer.

Definition at line 47 of file LSTMAttentionParameter.cs.

◆ num_output_ip

uint MyCaffe.param.LSTMAttentionParameter.num_output_ip

getset

Specifies the number of IP outputs for the layer. Note, when 0, no inner product is performed.

Definition at line 57 of file LSTMAttentionParameter.cs.

◆ weight_filler

FillerParameter MyCaffe.param.LSTMAttentionParameter.weight_filler

getset

Specifies the filler parameters for the weight filler.

Definition at line 78 of file LSTMAttentionParameter.cs.

The documentation for this class was generated from the following file:

C:/Data/Data/SS_Projects/Intelligence/GitHub/MyCaffe/MyCaffe/param.beta/LSTMAttentionParameter.cs

Public Member Functions

Static Public Member Functions

Properties

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ LSTMAttentionParameter()

Member Function Documentation

◆ Clone()

◆ Copy()

◆ FromProto()

◆ Load()

◆ ToProto()

Property Documentation

◆ bias_filler

◆ clipping_threshold

◆ enable_attention

◆ enable_clockwork_forgetgate_bias

◆ num_output

◆ num_output_ip

◆ weight_filler