Cpl::Text::Tokenizer::TextBlock Class Reference

This concrete class tokenizes a Text Block that has the following format: More...

Detailed Description

This concrete class tokenizes a Text Block that has the following format:

    parameter [(delimiter parameter)*] [terminator]

    parameter := { all printable characters except for QUOTE, DEL, TERM, and ESC }
    QUOTE     := { specified quote character (used to start/end TEXT STRINGS) }
    DEL       := { specified delimiter character }
    TERM      := { specified terminator character }
    ESC       := { specified Escape character used inside of TEXT STRINGS }

    The tokenizer replaces the delimiter and terminator characters with an '\0'
    character. The tokenizer supports the parameter fields being "text strings".
    In addition, any leading and trailing whitespace is removed from each
    parameter. Note: no whitespace is altered/removed from the contents of a
    text string.

    If the string being parsed has two DEL characters in row, then tokenized
    results will return an 'empty parameter' (a parameter will a zero string
    length) for this sequence.  The EXCEPTION to this rule is when the
    delimiter is the space character, then the tokenizer will never generated
    and/or detect any 'empty parameters'.

    The original string is "corrupted" since the tokenizer changes characters.
    Also, the tokenizer will shorten the string when dealing with 'text
    strings'.

    A TEXT STRING can contain any alphanumeric and/or punctuation characters
    including the QUOTE, DEL, TERM, and ESC characters.  A text string is
    always enclosed with beginning and ending QUOTE characters.  When a QUOTE
    character is encounter outside of a text string, it always starts a text
    string. To enter a literal QUOTE character within a text string, precede the
    QUOTE character with the ESC character (e.g., ").  To enter a literal ESC
    character within a text string, precede the character with ESC character
    (e.g., \). IMPORTANT NOTE: The QUOTE and ESC characters are REMOVED from
    the string when it is tokenized!

#include <TextBlock.h>

Public Member Functions
	TextBlock (char *string, char delimiter=',', char terminator=';', char quote='"', char escape='\\' )
	Constructor.
bool	isValidTokens () const noexcept
	Returns true if the string was successfully tokenized.
unsigned	numParameters () const noexcept
	Returns the number of parameter fields in the Text block.
const char *	getParameter (unsigned index) const noexcept
	Returns the Nth parameter (index starts with 0). Return a null pointer if index is out-of-range.
bool	isTerminated () const noexcept
	Returns true if the parsing stopped because the terminator character was encounter (vs. end-of-string)
const char *	remaining () const noexcept
	Returns a pointer to the portion of the string that has not been tokenized, i.e.

Protected Member Functions
void	removeWhiteSpace (char *startOfTokenPtr, char *firstNonSpacePtr, char *lastNonSpacePtr) noexcept
	Helpter method.

Constructor & Destructor Documentation

TextBlock()

Cpl::Text::Tokenizer::TextBlock::TextBlock	(	char *	string,
		char	delimiter = ',',
		char	terminator = ';',
		char	quote = '"',
		char	escape = '\\' )

Constructor.

Requires a pointer to the 'raw' string to be tokenized. Note: All of the parsing occurs in this method.

Member Function Documentation

getParameter()

const char * Cpl::Text::Tokenizer::TextBlock::getParameter ( unsigned index ) const

noexcept

Returns the Nth parameter (index starts with 0). Return a null pointer if index is out-of-range.

isTerminated()

bool Cpl::Text::Tokenizer::TextBlock::isTerminated ( ) const

inlinenoexcept

Returns true if the parsing stopped because the terminator character was encounter (vs. end-of-string)

isValidTokens()

bool Cpl::Text::Tokenizer::TextBlock::isValidTokens ( ) const

inlinenoexcept

Returns true if the string was successfully tokenized.

The string will fail to parse if it does not match the format/syntax described in the class definition.

THIS METHOD MUST BE CALLED BEFORE ANY OF THE FOLLOWING METHODS. THE PARSED TOKENS ARE ONLY VALID IF THIS METHOD RETURNS TRUE!

numParameters()

unsigned Cpl::Text::Tokenizer::TextBlock::numParameters ( ) const

inlinenoexcept

Returns the number of parameter fields in the Text block.

remaining()

const char * Cpl::Text::Tokenizer::TextBlock::remaining ( ) const

inlinenoexcept

Returns a pointer to the portion of the string that has not been tokenized, i.e.

the first character AFTER the terminator character.

removeWhiteSpace()

void Cpl::Text::Tokenizer::TextBlock::removeWhiteSpace ( char * startOfTokenPtr, char * firstNonSpacePtr, char * lastNonSpacePtr )

protectednoexcept

Helpter method.

---

The documentation for this class was generated from the following file:

• Cpl/Text/Tokenizer/TextBlock.h