Querylight TS Demo

KeywordTokenizer

Tokenizer that keeps the full input as a single token.

Reference Entry

API Reference · foundation · order 6

Tokenizer that keeps the full input as a single token.

Relevant APIs

Tokenizer that keeps the full input as a single token.

class KeywordTokenizer

```ts constructor(): KeywordTokenizer ```

```ts tokenize(text: string): string[] ``` Parameters: - `text: string`