Text parsing and tokenizing related functions.
- Source:
Classes
Methods
(static) calculateScore(match, positions) → {number}
Counts the not empty positions of match.
Parameters:
Name | Type | Description |
---|---|---|
match |
Array.<string> | The array to check. |
positions |
Array.<number> | The positions to check. |
- Source:
Returns:
The count of filled positions.
- Type
- number
(static) clean(text) → {string}
Clean text from diactrics, trims, removes double spaces and converts to lower case.
Parameters:
Name | Type | Description |
---|---|---|
text |
string | The text to be cleaned |
- Source:
Returns:
The cleaned text.
- Type
- string
(static) entityTrim(ent) → {Entity}
Trims a structured entity, usually produced by an entity recognizer. It moves the entity starting position and changes its string to its trimmed version.
Parameters:
Name | Type | Description |
---|---|---|
ent |
Entity | The entity to be processed. |
- Source:
Returns:
The trimmed entity.
- Type
- Entity
(static) generateGUID() → {string}
Generate a random GUID.
- Source:
Returns:
The generated GUID.
- Type
- string
(static) pad(n, width, zopt) → {string}
Adds zeroes to the left of n until the length width is reached.
Parameters:
Name | Type | Attributes | Default | Description |
---|---|---|---|---|
n |
number | The number to be padded. | ||
width |
number | The string length to be reached. | ||
z |
z |
<optional> |
"0" | The symbol to be used for padding. |
- Source:
Returns:
The padded value of n.
- Type
- string
(static) removeDiacritics(text) → {string}
Remove diatrics from a sentence, replacing them with not-diatrics representation.
Parameters:
Name | Type | Description |
---|---|---|
text |
string | The string to be processed |
- Source:
Returns:
The text without diatrics.
- Type
- string
(static) tokenize(text) → {Array.<string>}
Tokenize a string, splitting for non-words.
Parameters:
Name | Type | Description |
---|---|---|
text |
string | The string to be tokenized. |
- Source:
Returns:
The tokenized string.
- Type
- Array.<string>
(static) unique(list) → {Array.<string>}
Returns unique values of an array.
Parameters:
Name | Type | Description |
---|---|---|
list |
Array.<string> | The array to be processed. |
- Source:
Returns:
Unique values of list.
- Type
- Array.<string>