# Text tools API

> Chunk documents, redact Indian identifiers, count tokens, detect language.

Tools for working with text, especially in AI and retrieval pipelines. Send short text as ?text=, or POST it as text/plain, or as JSON {"text": ...} with the options beside it. Nothing you send is logged or kept. Each request counts as 10 against the rate limit.

Base URL `https://api.dharun.dev`. No API key. Answers are JSON, `{ "data": ..., "meta": { ... } }`, and errors are `{ "error": { "code", "message" } }` with a matching HTTP status. Rate limit: 600 requests a minute per client.

## Split a document into chunks

`GET /v1/text/chunk`, also `POST`

Splits at headings, paragraphs, sentences and words, in that order of preference, so chunks stay readable. Markdown chunks never cross a heading and carry their heading path. Each chunk is an exact slice of the input with its character offsets.

POST body: The document, plain text or Markdown. Send it as `text/plain`, or as JSON `{"text": "..."}` with the other options beside it.

What you send is never written to the log, and the answer is not cached.

Parameters:

- `text` (query): The text (or POST it).
- `size` (query): Largest chunk in characters, overlap included, 100 to 20000. Default 1000.
- `overlap` (query): Characters repeated from the previous chunk, up to half the size. Default a tenth of the size.
- `format` (query): Default auto: Markdown if the text has headings or code fences. One of: auto, markdown, text.

Example: https://api.dharun.dev/v1/text/chunk?size=200&overlap=20&text=%23+Setup%0A%0AInstall+Go.+Clone+the+repository.%0A%0A%23+Deploy%0A%0APush+to+main.

Response:

```json
{
  "data": {
    "format": "markdown",
    "size": 200,
    "chunks": [
      {
        "index": 0,
        "text": "# Setup\n\nInstall Go. Clone the repository.",
        "start": 0,
        "end": 42,
        "chars": 42,
        "heading": "Setup"
      },
      {
        "index": 1,
        "text": "# Deploy\n\nPush to main.",
        "start": 44,
        "end": 67,
        "chars": 23,
        "heading": "Deploy"
      }
    ]
  },
  "meta": {
    "total": 2,
    "note": "Sizes and offsets count characters (Unicode code points). Each chunk is exactly text[start:end]."
  }
}
```

## Redact Indian personal identifiers

`GET /v1/text/redact`, also `POST`

Masks Aadhaar numbers (Verhoeff-checked), PAN, GSTIN (checksum-checked), card numbers (Luhn-checked), Indian mobile numbers, email addresses, UPI IDs, vehicle numbers, voter IDs, passport numbers and IP addresses. Returns where each was found, never the values. Names and addresses are not detected.

POST body: The text to redact. Send it as `text/plain`, or as JSON `{"text": "..."}` with the other options beside it.

What you send is never written to the log, and the answer is not cached.

Parameters:

- `text` (query): The text (or POST it).
- `types` (query): Comma-separated kinds to redact; default all. Kinds: email, upi, gstin, card, aadhaar, pan, phone, vehicle, voter_id, passport, ip.
- `style` (query): label replaces with [PAN]; partial keeps the last digits, as on a bank statement. Default label. One of: label, partial.

Example: https://api.dharun.dev/v1/text/redact?text=Call+98765+43210%2C+PAN+ABCPK1234F

Response:

```json
{
  "data": {
    "text": "Call [PHONE], PAN [PAN]",
    "findings": [
      {
        "type": "phone",
        "start": 5,
        "end": 16
      },
      {
        "type": "pan",
        "start": 22,
        "end": 32
      }
    ],
    "counts": {
      "pan": 1,
      "phone": 1
    }
  },
  "meta": {
    "note": "Finds identifiers by their structure and check digits. Names and street addresses are not detected."
  }
}
```

## Count tokens

`GET /v1/text/tokens`, also `POST`

Exact token counts with OpenAI tokenizers, by encoding or model name. Other vendors do not publish their tokenizers, so their counts are not offered.

POST body: The text to count. Send it as `text/plain`, or as JSON `{"text": "..."}` with the other options beside it.

What you send is never written to the log, and the answer is not cached.

Parameters:

- `text` (query): The text (or POST it).
- `model` (query): Model name, such as gpt-4o or gpt-4, or an encoding: o200k_base, cl100k_base, p50k_base. Default o200k_base.

Example: https://api.dharun.dev/v1/text/tokens?model=gpt-4o&text=%E0%AE%B5%E0%AE%A3%E0%AE%95%E0%AF%8D%E0%AE%95%E0%AE%AE%E0%AF%8D%2C+how+many+tokens+is+this%3F

Response, with lists and long text shortened:

```json
{
  "data": {
    "tokens": 10,
    "encoding": "o200k_base",
    "models": [
      "gpt-4o",
      "gpt-4o-mini",
      "gpt-4.1"
    ],
    "characters": 33,
    "words": 6,
    "bytes": 47
  },
  "meta": {
    "note": "Counted with OpenAI's vocabulary for the encoding, treating special tokens as text. Chat requests add a few tokens per message on top."
  }
}
```

## Detect the language

`GET /v1/text/language`, also `POST`

Language and script, with a confidence. Covers about 80 languages including Hindi, Bengali, Marathi, Gujarati, Punjabi, Tamil, Telugu, Kannada, Malayalam, Odia and Urdu. Romanised Indian text (Hinglish, Tanglish) is flagged as such.

POST body: The text. Send it as `text/plain`, or as JSON `{"text": "..."}` with the other options beside it.

What you send is never written to the log, and the answer is not cached.

Parameters:

- `text` (query): The text (or POST it).

Example: https://api.dharun.dev/v1/text/language?text=%E0%AE%85%E0%AE%95%E0%AE%B0+%E0%AE%AE%E0%AF%81%E0%AE%A4%E0%AE%B2+%E0%AE%8E%E0%AE%B4%E0%AF%81%E0%AE%A4%E0%AF%8D%E0%AE%A4%E0%AF%86%E0%AE%B2%E0%AF%8D%E0%AE%B2%E0%AE%BE%E0%AE%AE%E0%AF%8D+%E0%AE%86%E0%AE%A4%E0%AE%BF+%E0%AE%AA%E0%AE%95%E0%AE%B5%E0%AE%A9%E0%AF%8D+%E0%AE%AE%E0%AF%81%E0%AE%A4%E0%AE%B1%E0%AF%8D%E0%AE%B1%E0%AF%87+%E0%AE%89%E0%AE%B2%E0%AE%95%E0%AF%81

Response:

```json
{
  "data": {
    "code": "tam",
    "code2": "ta",
    "name": "Tamil",
    "script": "Tamil",
    "confidence": 1,
    "reliable": true
  },
  "meta": {
    "note": "About 80 languages from their letters. Short texts are uncertain; Romanised Indian languages are flagged but not identified."
  }
}
```

## Guides

- [Remove Aadhaar, PAN and phone numbers before sending text to an LLM](https://api.dharun.dev/guides/redact-before-llm.md): Mask Indian personal identifiers in text before it reaches a language model or a log, with check-digit validation so ordinary numbers are left alone.