Specifies the parameters for the MultiHeadAttentionInterpLayer (Interpretable Multi-Head Attention Layer).
More...

Inheritance diagram for MyCaffe.param.tft.MultiHeadAttentionInterpParameter:

Public Member Functions
	MultiHeadAttentionInterpParameter ()
	Constructor for the parameter. More...

override object	Load (System.IO.BinaryReader br, bool bNewInstance=true)
	Load the parameter from a binary reader. More...

override void	Copy (LayerParameterBase src)
	Copy on parameter to another. More...

override LayerParameterBase	Clone ()
	Creates a new copy of this instance of the parameter. More...

override RawProto	ToProto (string strName)
	Convert the parameter into a RawProto. More...

Public Member Functions inherited from MyCaffe.param.LayerParameterBase
	LayerParameterBase ()
	Constructor for the parameter. More...

virtual string	PrepareRunModelInputs ()
	This method gives derivative classes a chance specify model inputs required by the run model. More...

virtual void	PrepareRunModel (LayerParameter p)
	This method gives derivative classes a chance to prepare the layer for a run-model. More...

void	Save (BinaryWriter bw)
	Save this parameter to a binary writer. More...

abstract object	Load (BinaryReader br, bool bNewInstance=true)
	Load the parameter from a binary reader. More...

Public Member Functions inherited from MyCaffe.basecode.BaseParameter
	BaseParameter ()
	Constructor for the parameter. More...

virtual bool	Compare (BaseParameter p)
	Compare this parameter to another parameter. More...

Static Public Member Functions
static MultiHeadAttentionInterpParameter	FromProto (RawProto rp)
	Parses the parameter from a RawProto. More...

Static Public Member Functions inherited from MyCaffe.basecode.BaseParameter
static double	ParseDouble (string strVal)
	Parse double values using the US culture if the decimal separator = '.', then using the native culture, and if then lastly trying the US culture to handle prototypes containing '.' as the separator, yet parsed in a culture that does not use '.' as a decimal. More...

static bool	TryParse (string strVal, out double df)
	Parse double values using the US culture if the decimal separator = '.', then using the native culture, and if then lastly trying the US culture to handle prototypes containing '.' as the separator, yet parsed in a culture that does not use '.' as a decimal. More...

static float	ParseFloat (string strVal)
	Parse float values using the US culture if the decimal separator = '.', then using the native culture, and if then lastly trying the US culture to handle prototypes containing '.' as the separator, yet parsed in a culture that does not use '.' as a decimal. More...

static bool	TryParse (string strVal, out float f)
	Parse doufloatble values using the US culture if the decimal separator = '.', then using the native culture, and if then lastly trying the US culture to handle prototypes containing '.' as the separator, yet parsed in a culture that does not use '.' as a decimal. More...

Properties
bool	enable_self_attention `[getset]`
	Specifies to enable self attention (one input, default = true). More...

uint	num_historical_steps `[getset]`
	Specifies the number of historical steps More...

uint	num_future_steps `[getset]`
	Specifies the number of future steps More...

uint	embed_dim `[getset]`
	Specifies the state size corresponding to both the input and output sizes. More...

uint	num_heads `[getset]`
	Specifies number of attention heads used in the multi-attention. More...

bool	enable_noise `[getset]`
	Enable/disable noise in the inner-product layer (default = false). More...

double	sigma_init `[getset]`
	Specifies the initialization value for the sigma weight and sigma bias used when 'enable_noise' = true. More...

FillerParameter	weight_filler `[getset]`
	The filler for the weights. More...

FillerParameter	bias_filler `[getset]`
	The filler for the bias. More...

Additional Inherited Members
Public Types inherited from MyCaffe.param.LayerParameterBase
enum	LABEL_TYPE { NONE , SINGLE , MULTIPLE , ONLY_ONE }
	Defines the label type. More...

Detailed Description

Specifies the parameters for the MultiHeadAttentionInterpLayer (Interpretable Multi-Head Attention Layer).

The Multi-Headed Attention layer learns long-term relationships across different time-steps. This version of the layer is modified to enhance explainability. On this modification, the 'values' signal is shared across all heads - the additive aggregation is employed across all heads. According to the paper by Lim et al., each head can learn different temporal patterns, while attending to a common set of input features which can be interpreted as a simple ensemble over attention weights into a combined matrix, which compared to the original multi-head attention matrix, yields an increased representation capacity in an efficient way.

See also: Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting by Bryan Lim, Sercan O. Arik, Nicolas Loeff, and Tomas Pfister, 2019, arXiv 1912.09363; Github - PlaytikaOSS/tft-torch by Playtika Research, 2021.; Github - PlaytikaOSS/tft-torch tft.py by Playtika Research, 2021.

Definition at line 27 of file MultiHeadAttentionInterpParameter.cs.

Constructor & Destructor Documentation

◆ MultiHeadAttentionInterpParameter()

MyCaffe.param.tft.MultiHeadAttentionInterpParameter.MultiHeadAttentionInterpParameter ( )

Constructor for the parameter.

Definition at line 40 of file MultiHeadAttentionInterpParameter.cs.

Member Function Documentation

◆ Clone()

override LayerParameterBase MyCaffe.param.tft.MultiHeadAttentionInterpParameter.Clone ( )

virtual

Creates a new copy of this instance of the parameter.

Returns: A new instance of this parameter is returned.

Implements MyCaffe.param.LayerParameterBase.

Definition at line 174 of file MultiHeadAttentionInterpParameter.cs.

◆ Copy()

override void MyCaffe.param.tft.MultiHeadAttentionInterpParameter.Copy ( LayerParameterBase src )

virtual

Copy on parameter to another.

Parameters

src	Specifies the parameter to copy.

Implements MyCaffe.param.LayerParameterBase.

Definition at line 152 of file MultiHeadAttentionInterpParameter.cs.

◆ FromProto()

static MultiHeadAttentionInterpParameter MyCaffe.param.tft.MultiHeadAttentionInterpParameter.FromProto ( RawProto rp )

static

Parses the parameter from a RawProto.

Parameters

rp	Specifies the RawProto to parse.

Returns: A new instance of the parameter is returned.

Definition at line 217 of file MultiHeadAttentionInterpParameter.cs.

◆ Load()

override object MyCaffe.param.tft.MultiHeadAttentionInterpParameter.Load	(	System.IO.BinaryReader	br,
		bool	bNewInstance = `true`
	)

Load the parameter from a binary reader.

Parameters

br	Specifies the binary reader.
bNewInstance	When true a new instance is created (the default), otherwise the existing instance is loaded from the binary reader.

Returns: Returns an instance of the parameter.

Definition at line 140 of file MultiHeadAttentionInterpParameter.cs.

◆ ToProto()

override RawProto MyCaffe.param.tft.MultiHeadAttentionInterpParameter.ToProto ( string strName )

virtual

Convert the parameter into a RawProto.

Parameters

strName Specifies the name to associate with the RawProto.

Returns: The new RawProto is returned.

Implements MyCaffe.basecode.BaseParameter.

Definition at line 186 of file MultiHeadAttentionInterpParameter.cs.

Property Documentation

◆ bias_filler

FillerParameter MyCaffe.param.tft.MultiHeadAttentionInterpParameter.bias_filler

getset

The filler for the bias.

Definition at line 133 of file MultiHeadAttentionInterpParameter.cs.

◆ embed_dim

uint MyCaffe.param.tft.MultiHeadAttentionInterpParameter.embed_dim

getset

Specifies the state size corresponding to both the input and output sizes.

Definition at line 78 of file MultiHeadAttentionInterpParameter.cs.

◆ enable_noise

bool MyCaffe.param.tft.MultiHeadAttentionInterpParameter.enable_noise

getset

Enable/disable noise in the inner-product layer (default = false).

When enabled, noise is only used during the training phase.

Definition at line 101 of file MultiHeadAttentionInterpParameter.cs.

◆ enable_self_attention

bool MyCaffe.param.tft.MultiHeadAttentionInterpParameter.enable_self_attention

getset

Specifies to enable self attention (one input, default = true).

Definition at line 48 of file MultiHeadAttentionInterpParameter.cs.

◆ num_future_steps

uint MyCaffe.param.tft.MultiHeadAttentionInterpParameter.num_future_steps

getset

Specifies the number of future steps

Definition at line 68 of file MultiHeadAttentionInterpParameter.cs.

◆ num_heads

uint MyCaffe.param.tft.MultiHeadAttentionInterpParameter.num_heads

getset

Specifies number of attention heads used in the multi-attention.

Definition at line 88 of file MultiHeadAttentionInterpParameter.cs.

◆ num_historical_steps

uint MyCaffe.param.tft.MultiHeadAttentionInterpParameter.num_historical_steps

getset

Specifies the number of historical steps

Definition at line 58 of file MultiHeadAttentionInterpParameter.cs.

◆ sigma_init

double MyCaffe.param.tft.MultiHeadAttentionInterpParameter.sigma_init

getset

Specifies the initialization value for the sigma weight and sigma bias used when 'enable_noise' = true.

Definition at line 111 of file MultiHeadAttentionInterpParameter.cs.

◆ weight_filler

FillerParameter MyCaffe.param.tft.MultiHeadAttentionInterpParameter.weight_filler

getset

The filler for the weights.

Definition at line 122 of file MultiHeadAttentionInterpParameter.cs.

The documentation for this class was generated from the following file:

C:/Data/Data/SS_Projects/Intelligence/GitHub/MyCaffe/MyCaffe/param.tft/MultiHeadAttentionInterpParameter.cs

Public Member Functions

Static Public Member Functions

Properties

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ MultiHeadAttentionInterpParameter()

Member Function Documentation

◆ Clone()

◆ Copy()

◆ FromProto()

◆ Load()

◆ ToProto()

Property Documentation

◆ bias_filler

◆ embed_dim

◆ enable_noise

◆ enable_self_attention

◆ num_future_steps

◆ num_heads

◆ num_historical_steps

◆ sigma_init

◆ weight_filler