Querylight TS Demo

SplittingTokenizer

Whitespace tokenizer for ordinary text analysis.

Back to docs search

Reference Entry

SplittingTokenizer

API Reference · foundation · order 7

Whitespace tokenizer for ordinary text analysis.

Relevant APIs

SplittingTokenizer

Tags

api-reference classes

SplittingTokenizer

Whitespace tokenizer for ordinary text analysis.

Declaration

class SplittingTokenizer

Constructors

constructor

```ts constructor(): SplittingTokenizer ```

Members

tokenize

```ts tokenize(text: string): string[] ``` Parameters: - `text: string`