Specifies the parameters for the TokenizedDataPairsLayer. More...

Inheritance diagram for MyCaffe.param.gpt.TokenizedDataPairsParameter:

Public Member Functions
	TokenizedDataPairsParameter ()
	Constructor for the parameter. More...

override object	Load (System.IO.BinaryReader br, bool bNewInstance=true)
	Load the parameter from a binary reader. More...

override void	Copy (LayerParameterBase src)
	Copy on parameter to another. More...

override LayerParameterBase	Clone ()
	Creates a new copy of this instance of the parameter. More...

override RawProto	ToProto (string strName)
	Convert the parameter into a RawProto. More...

Public Member Functions inherited from MyCaffe.param.gpt.TokenizedDataParameter
	TokenizedDataParameter ()
	Constructor for the parameter. More...

override object	Load (System.IO.BinaryReader br, bool bNewInstance=true)
	Load the parameter from a binary reader. More...

override void	Copy (LayerParameterBase src)
	Copy on parameter to another. More...

override LayerParameterBase	Clone ()
	Creates a new copy of this instance of the parameter. More...

override RawProto	ToProto (string strName)
	Convert the parameter into a RawProto. More...

Public Member Functions inherited from MyCaffe.param.LayerParameterBase
	LayerParameterBase ()
	Constructor for the parameter. More...

virtual string	PrepareRunModelInputs ()
	This method gives derivative classes a chance specify model inputs required by the run model. More...

virtual void	PrepareRunModel (LayerParameter p)
	This method gives derivative classes a chance to prepare the layer for a run-model. More...

void	Save (BinaryWriter bw)
	Save this parameter to a binary writer. More...

abstract object	Load (BinaryReader br, bool bNewInstance=true)
	Load the parameter from a binary reader. More...

Public Member Functions inherited from MyCaffe.basecode.BaseParameter
	BaseParameter ()
	Constructor for the parameter. More...

virtual bool	Compare (BaseParameter p)
	Compare this parameter to another parameter. More...

Static Public Member Functions
static new TokenizedDataPairsParameter	FromProto (RawProto rp)
	Parses the parameter from a RawProto. More...

Static Public Member Functions inherited from MyCaffe.param.gpt.TokenizedDataParameter
static TokenizedDataParameter	FromProto (RawProto rp)
	Parses the parameter from a RawProto. More...

Static Public Member Functions inherited from MyCaffe.basecode.BaseParameter
static double	ParseDouble (string strVal)
	Parse double values using the US culture if the decimal separator = '.', then using the native culture, and if then lastly trying the US culture to handle prototypes containing '.' as the separator, yet parsed in a culture that does not use '.' as a decimal. More...

static bool	TryParse (string strVal, out double df)
	Parse double values using the US culture if the decimal separator = '.', then using the native culture, and if then lastly trying the US culture to handle prototypes containing '.' as the separator, yet parsed in a culture that does not use '.' as a decimal. More...

static float	ParseFloat (string strVal)
	Parse float values using the US culture if the decimal separator = '.', then using the native culture, and if then lastly trying the US culture to handle prototypes containing '.' as the separator, yet parsed in a culture that does not use '.' as a decimal. More...

static bool	TryParse (string strVal, out float f)
	Parse doufloatble values using the US culture if the decimal separator = '.', then using the native culture, and if then lastly trying the US culture to handle prototypes containing '.' as the separator, yet parsed in a culture that does not use '.' as a decimal. More...

Properties
string	target `[getset]`
	Specifies the data source based on the INPUT_TYPE used. Each dataset has both a training and testing data source and target consisting of matching lines. For example, when using the TEXT_FILE input type, the source may be the eng.txt file of english phraes and the target may be the fra.txt file of french phrases. More...

string	vocab_data_url `[getset]`
	Specifies the URL to the vocabulary data file used with the SENTENCEPIECE vocabulary type. This pre-created vocabulary file is also created using the Python SentencePieceProcess. More...

string	vocab_data_dst_file `[getset]`
	Specifies the destination file where the vocabulary data file data is downloaded. This pre-created vocabulary file is also created using the Python SentencePieceProcess. More...

string	source_vocab_file `[getset]`
	Specifies the source vocabulary file used with the SENTENCEPIECE vocabulary type. The vocabulary file is created using the Python SentencePieceProcess. More...

string	target_vocab_file `[getset]`
	Specifies the target vocabulary file used with the SENTENCEPIECE vocabulary type. The vocabulary file is created using the Python SentencePieceProcess. More...

int	max_load `[getset]`
	Specifies the maximum items to load - primarily used for testing. More...

Properties inherited from MyCaffe.param.gpt.TokenizedDataParameter
PythonParameter	python_param `[getset]`
	Specifies the PythonParameter used by the python implementation of the TokenizedDataPairsLayer, otherwise this is null. More...

int?	seed `[getset]`
	Specifies the seed used to initialize the random number generator (normally only for testing). More...

INPUT_TYPE	input_type `[getset]`
	Specifies data source input type. More...

VOCABULARY_TYPE	vocabulary_type `[getset]`
	Specifies the vocabulary type to use. More...

SAMPLE_METHOD	sample_method `[getset]`
	Specifies the sampling method used when post processing logits (default = ARGMAX). More...

string	source `[getset]`
	Specifies the data source based on the INPUT_TYPE used. Each dataset has both a training and testing data source. More...

string	debug_index_file `[getset]`
	Specifies an optional data index file used for debugging only. More...

uint	batch_size `[getset]`
	The number of heads used. More...

uint	block_size `[getset]`
	Specifies size of the block. More...

Additional Inherited Members
Public Types inherited from MyCaffe.param.gpt.TokenizedDataParameter
enum	VOCABULARY_TYPE { CHARACTER , WORD , SENTENCEPIECE , CUSTOM }
	Defines the vocabulary type to use. More...

enum	SAMPLE_METHOD { ARGMAX , PROBABILITY }
	Defines the sampling method used. More...

enum	INPUT_TYPE { TEXT_FILE , CUSTOM }
	Defines the input type used. More...

Public Types inherited from MyCaffe.param.LayerParameterBase
enum	LABEL_TYPE { NONE , SINGLE , MULTIPLE , ONLY_ONE }
	Defines the label type. More...

Protected Attributes inherited from MyCaffe.param.gpt.TokenizedDataParameter
PythonParameter	m_pythonParam = new PythonParameter()
	Python layer implementations use this parameter for Python specific settings such as the location of the runtime. More...

Detailed Description

Specifies the parameters for the TokenizedDataPairsLayer.

Definition at line 16 of file TokenizedDataPairsParameter.cs.

Constructor & Destructor Documentation

◆ TokenizedDataPairsParameter()

MyCaffe.param.gpt.TokenizedDataPairsParameter.TokenizedDataPairsParameter ( )

Constructor for the parameter.

Definition at line 26 of file TokenizedDataPairsParameter.cs.

Member Function Documentation

◆ Clone()

override LayerParameterBase MyCaffe.param.gpt.TokenizedDataPairsParameter.Clone ( )

virtual

Creates a new copy of this instance of the parameter.

Returns: A new instance of this parameter is returned.

Implements MyCaffe.param.LayerParameterBase.

Definition at line 121 of file TokenizedDataPairsParameter.cs.

◆ Copy()

override void MyCaffe.param.gpt.TokenizedDataPairsParameter.Copy ( LayerParameterBase src )

virtual

Copy on parameter to another.

Parameters

src	Specifies the parameter to copy.

Implements MyCaffe.param.LayerParameterBase.

Definition at line 104 of file TokenizedDataPairsParameter.cs.

◆ FromProto()

static new TokenizedDataPairsParameter MyCaffe.param.gpt.TokenizedDataPairsParameter.FromProto ( RawProto rp )

static

Parses the parameter from a RawProto.

Parameters

rp	Specifies the RawProto to parse.

Returns: A new instance of the parameter is returned.

Definition at line 157 of file TokenizedDataPairsParameter.cs.

◆ Load()

override object MyCaffe.param.gpt.TokenizedDataPairsParameter.Load	(	System.IO.BinaryReader	br,
		bool	bNewInstance = `true`
	)

Load the parameter from a binary reader.

Parameters

br	Specifies the binary reader.
bNewInstance	When true a new instance is created (the default), otherwise the existing instance is loaded from the binary reader.

Returns: Returns an instance of the parameter.

Definition at line 92 of file TokenizedDataPairsParameter.cs.

◆ ToProto()

override RawProto MyCaffe.param.gpt.TokenizedDataPairsParameter.ToProto ( string strName )

virtual

Convert the parameter into a RawProto.

Parameters

strName Specifies the name to associate with the RawProto.

Returns: The new RawProto is returned.

Implements MyCaffe.basecode.BaseParameter.

Definition at line 133 of file TokenizedDataPairsParameter.cs.

Property Documentation

◆ max_load

int MyCaffe.param.gpt.TokenizedDataPairsParameter.max_load

getset

Specifies the maximum items to load - primarily used for testing.

Definition at line 85 of file TokenizedDataPairsParameter.cs.

◆ source_vocab_file

string MyCaffe.param.gpt.TokenizedDataPairsParameter.source_vocab_file

getset

Specifies the source vocabulary file used with the SENTENCEPIECE vocabulary type. The vocabulary file is created using the Python SentencePieceProcess.

Definition at line 65 of file TokenizedDataPairsParameter.cs.

◆ target

string MyCaffe.param.gpt.TokenizedDataPairsParameter.target

getset

Specifies the data source based on the INPUT_TYPE used. Each dataset has both a training and testing data source and target consisting of matching lines.
For example, when using the TEXT_FILE input type, the source may be the eng.txt file of english phraes and the target may be the fra.txt file of french phrases.

Definition at line 36 of file TokenizedDataPairsParameter.cs.

◆ target_vocab_file

string MyCaffe.param.gpt.TokenizedDataPairsParameter.target_vocab_file

getset

Specifies the target vocabulary file used with the SENTENCEPIECE vocabulary type. The vocabulary file is created using the Python SentencePieceProcess.

Definition at line 75 of file TokenizedDataPairsParameter.cs.

◆ vocab_data_dst_file

string MyCaffe.param.gpt.TokenizedDataPairsParameter.vocab_data_dst_file

getset

Specifies the destination file where the vocabulary data file data is downloaded. This pre-created vocabulary file is also created using the Python SentencePieceProcess.

Definition at line 55 of file TokenizedDataPairsParameter.cs.

◆ vocab_data_url

string MyCaffe.param.gpt.TokenizedDataPairsParameter.vocab_data_url

getset

Specifies the URL to the vocabulary data file used with the SENTENCEPIECE vocabulary type. This pre-created vocabulary file is also created using the Python SentencePieceProcess.

Definition at line 46 of file TokenizedDataPairsParameter.cs.

The documentation for this class was generated from the following file:

C:/Data/Data/SS_Projects/Intelligence/GitHub/MyCaffe/MyCaffe/param.gpt/TokenizedDataPairsParameter.cs

Public Member Functions

Static Public Member Functions

Properties

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ TokenizedDataPairsParameter()

Member Function Documentation

◆ Clone()

◆ Copy()

◆ FromProto()

◆ Load()

◆ ToProto()

Property Documentation

◆ max_load

◆ source_vocab_file

◆ target

◆ target_vocab_file

◆ vocab_data_dst_file

◆ vocab_data_url