> For a complete page index of the Captain API documentation, fetch https://docs.filesearch.dev/llms.txt?excludeSpec=true

# Index URL

POST https://api.filesearch.dev/v2/collections/{collection_name}/index/url
Content-Type: application/json

Index documents from public URL(s) into a collection.

Accepts either a single `url` string or a `urls` array of strings.
Documents are downloaded from the provided URLs and processed through
the same pipeline as cloud storage indexing.

Supported formats: PDF, TXT, DOCX, CSV, XLSX, and other document types.

Headers:

* Authorization: Bearer \{api\_key} - Captain API key for authentication
* X-Organization-ID: Organization UUID
* Idempotency-Key: UUID for request deduplication (optional)

Args:
collection\_name: Name of the collection (path parameter)
body: URL configuration with url or urls

Returns:
\{ job\_id, status: "pending" }

Reference: https://docs.filesearch.dev/reference/indexing/urls/create

## Request

### Path parameters

- `collection_name` (string, required)

### Headers

- `authorization` (string, optional, nullable)

### Body (application/json)

This endpoint expects an IndexURLRequest.

- `processing_type` (enum, required) — Document processing type. 'advanced' uses agentic OCR with AI-enhanced extraction for complex layouts, tables, figures, charts, and documents containing images. 'basic' provides reliable OCR optimized for general document indexing and high-volume processing. 'auto' inspects each PDF and uses advanced only when it has photos, drawings, tables or scanned pages (logos and signatures do not count), otherwise basic; each file bills at the tier it used.
  - Allowed values: `advanced`, `basic`, `auto`
- `custom_metadata` (map from string to IndexUrlRequestCustomMetadata, optional, nullable) — Custom metadata to attach to all indexed chunks. Keys must be strings. Values: str, int, float, bool, or List[str].
- `parsing_script` (string, optional, nullable) — Relative path to a JS parsing script for JSON files (e.g. 'research/paper-parser'). When provided, .json files are processed through a sandboxed V8 isolate. Without this, .json files are indexed as raw text.
- `transcription_language` (string, optional, nullable) — AWS Transcribe language code for the spoken audio (e.g. 'es-US', 'pt-BR'). Omit to auto-detect per file. Video and audio files only. Supported codes: https://docs.aws.amazon.com/transcribe/latest/dg/supported-languages.html
- `url` (string, optional, nullable) — A single public URL to a hosted document. Supported types: PDF, DOCX, DOC, XLSX, XLS, CSV, TSV, TXT, MD, JSON, YAML, YML, PNG, JPG, JPEG, GIF, BMP, TIFF. Provide either 'url' or 'urls', not both.
- `urls` (list of string, optional, nullable) — A list of public URLs to hosted documents. Provide either 'url' or 'urls', not both.
- `mask_pii` (boolean, optional, default: false) — When true, detected PII (emails, phone numbers, SSNs, credit cards, names, and locations) is masked in the parsed content before it is embedded and stored — replaced with entity tags like \<PERSON> and \<EMAIL\_ADDRESS>. For images (including images embedded in PDFs), PII text visible in the image is also pixel-redacted. Opt-in; defaults to false, which leaves content unchanged.
- `pii_engine` (enum, optional, default: kev) — Masking engine for this job. `kev` (default): the Kev classifier model on Captain's infrastructure; supports `pii_fields` and `pii_instructions`; no fallback, so a file kev cannot mask fails to index and the rest of the job continues. `jev`: TypeSafe's hosted Jev model, slightly more accurate, best-effort availability; supports custom fields and instructions and may list fallbacks in `pii_fallback`. `presidio`: the legacy pattern engine, built-in categories only (rejects `pii_fields` or `pii_instructions` with 422), never a fallback. Requires `mask_pii: true`. The earlier names `captain-jev` and `captain-presidio` keep working as before; new requests should use `kev`, `jev` or `presidio`.
  - Allowed values: `kev`, `jev`, `presidio`, `captain-jev`, `captain-presidio`
- `pii_fallback` (IndexUrlRequestPiiFallback, optional) — Ordered fallback engines for `jev`, as `{"engines": [...], "retry_budget_seconds": n}`. Omitted or an empty `engines` list means no fallback. Only valid with `pii_engine: "jev"`; with `kev` or `presidio` a non-empty list answers 422. On the multipart file upload endpoint, send it as a JSON string form field. With the earlier engine name `captain-jev` it is `true` (default, fall back to `captain-presidio`) or `false`.
- `pii_fields` (list of PIIField, optional, nullable) — Additional PII categories to mask for this job, on top of the built-in set. Requires mask_pii to be true. Up to 20 fields. Each masked value is replaced with the field's tag in angle brackets and appears in the masking report under the field's name.
- `pii_instructions` (string, optional, nullable) — Plain-language guidance that steers what counts as personal data for this job, for the built-in categories and any pii_fields. For example: 'Names of hospital staff may stay; patient names, record numbers and bed assignments must be masked.' Requires mask_pii to be true. Up to 1,000 characters.

## Response

### 200

Successful Response

- `job_id` (string, required)
- `status` (string, optional, default: pending)
- `custom_metadata` (map from string to any, optional, nullable) — The custom_metadata Captain accepted for this job, echoed back as validated. Null when none was supplied.

## Types

### IndexUrlRequestCustomMetadata

- string
- integer
- double
- boolean
- list of string

### IndexUrlRequestPiiFallback

Ordered fallback engines for `jev`, as `{"engines": [...], "retry_budget_seconds": n}`. Omitted or an empty `engines` list means no fallback. Only valid with `pii_engine: "jev"`; with `kev` or `presidio` a non-empty list answers 422. On the multipart file upload endpoint, send it as a JSON string form field. With the earlier engine name `captain-jev` it is `true` (default, fall back to `captain-presidio`) or `false`.

- PiiFallbackPolicy: PiiFallbackPolicy
- boolean

### PIIField

A customer-defined PII category to mask in addition to the built-in ones.

- `name` (string, required) — Tag for this category. Uppercase letters, digits and underscores, 2 to 40 characters, unique within the request and not one of the built-in tags. Masked values are replaced with the tag in angle brackets (a field named EMPLOYEE\_ID produces \<EMPLOYEE\_ID>) and the masking report lists them under this name.
- `description` (string, required) — What the category covers, in plain language, as you would explain it to a colleague. This is what decides whether a value is masked, so say what the values look like and where they appear.
- `examples` (list of string, optional, nullable) — Up to 10 sample values, used only to make the description concrete. They are not a pattern: values that do not look like the examples are still masked when the description covers them.

### PiiFallbackPolicy

Fallback policy for `pii_engine: "jev"`. If every engine fails, the file fails to index and the rest of the job continues.

- `engines` (list of enum, optional, default: []) — Engines to try, in order, when `jev` fails with a rate limit, timeout, capacity limit, provider error or invalid answer. `jev-openrouter` and `jev-ai-gateway` are the same Jev model served through OpenRouter and Vercel AI Gateway. Custom fields and instructions apply on every engine. `presidio` is not allowed, and an engine may appear only once.
  - Allowed values: `jev-openrouter`, `jev-ai-gateway`, `kev`
- `retry_budget_seconds` (integer, optional, default: 60) — Extra retry and wait time per engine, for each piece of a file, before moving to the next engine. Time spent on successful answers does not count. 0 means one attempt per engine. Normal request timeouts still apply.

## Examples

### Single URL

**Request**

```json
{
  "processing_type": "advanced",
  "url": "https://example.com/documents/report.pdf"
}
```

**Response**

```json
{
  "job_id": "job_url_abc123",
  "status": "pending"
}
```

**SDK Code**

```python Python REST
import requests

BASE_URL = "https://api.filesearch.dev"
API_KEY = "your_api_key"

headers = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json"
}

response = requests.post(
    f"{BASE_URL}/v2/collections/my_documents/index/url",
    headers=headers,
    json={
        "url": "https://example.com/documents/report.pdf",
        "processing_type": "advanced"
    },
    timeout=60.0
)

if response.status_code in [200, 201]:
    data = response.json()
    print(f"Job started! ID: {data['job_id']}")
else:
    print(f"Error: {response.status_code}")
```

```typescript TypeScript REST
const BASE_URL = "https://api.filesearch.dev";
const API_KEY = "your_api_key";

const response = await fetch(
    `${BASE_URL}/v2/collections/my_documents/index/url`,
    {
        method: "POST",
        headers: {
            "Authorization": `Bearer ${API_KEY}`,
            "Content-Type": "application/json"
        },
        body: JSON.stringify({
            url: "https://example.com/documents/report.pdf",
            processing_type: "advanced"
        })
    }
);

if (response.ok) {
    const data = await response.json();
    console.log(`Job started! ID: ${data.job_id}`);
} else {
    console.error(`Error: ${response.status}`);
}
```

### Multiple URLs

**Request**

```json
{
  "processing_type": "advanced",
  "urls": [
    "https://example.com/documents/report.pdf",
    "https://example.com/documents/memo.txt",
    "https://example.com/documents/data.csv"
  ]
}
```

**Response**

```json
{
  "job_id": "job_url_abc456",
  "status": "pending"
}
```

**SDK Code**

```python Python REST
import requests

BASE_URL = "https://api.filesearch.dev"
API_KEY = "your_api_key"

headers = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json"
}

response = requests.post(
    f"{BASE_URL}/v2/collections/my_documents/index/url",
    headers=headers,
    json={
        "urls": [
            "https://example.com/documents/report.pdf",
            "https://example.com/documents/memo.txt",
            "https://example.com/documents/data.csv"
        ],
        "processing_type": "advanced"
    },
    timeout=60.0
)

if response.status_code in [200, 201]:
    data = response.json()
    print(f"Job started! ID: {data['job_id']}")
else:
    print(f"Error: {response.status_code}")
```

```typescript TypeScript REST
const BASE_URL = "https://api.filesearch.dev";
const API_KEY = "your_api_key";

const response = await fetch(
    `${BASE_URL}/v2/collections/my_documents/index/url`,
    {
        method: "POST",
        headers: {
            "Authorization": `Bearer ${API_KEY}`,
            "Content-Type": "application/json"
        },
        body: JSON.stringify({
            urls: [
                "https://example.com/documents/report.pdf",
                "https://example.com/documents/memo.txt",
                "https://example.com/documents/data.csv"
            ],
            processing_type: "advanced"
        })
    }
);

if (response.ok) {
    const data = await response.json();
    console.log(`Job started! ID: ${data.job_id}`);
} else {
    console.error(`Error: ${response.status}`);
}
```

### Web Page (auto-scraped)

**Request**

```json
{
  "processing_type": "advanced",
  "url": "https://stripe.com/about"
}
```

**Response**

```json
{
  "job_id": "job_url_web_789",
  "status": "pending"
}
```

**SDK Code**

```python Python REST
import requests
import json

BASE_URL = "https://api.filesearch.dev"
API_KEY = "your_api_key"

headers = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json"
}

# Web pages are auto-detected and scraped — text and images are extracted
response = requests.post(
    f"{BASE_URL}/v2/collections/my_documents/index/url",
    headers=headers,
    json={
        "url": "https://stripe.com/about",
        "processing_type": "advanced"
    },
    timeout=60.0
)

data = response.json()
print(json.dumps(data, indent=2))
```

```typescript TypeScript REST
const BASE_URL = "https://api.filesearch.dev";
const API_KEY = "your_api_key";

// Web pages are auto-detected and scraped — text and images are extracted
const response = await fetch(
    `${BASE_URL}/v2/collections/my_documents/index/url`,
    {
        method: "POST",
        headers: {
            "Authorization": `Bearer ${API_KEY}`,
            "Content-Type": "application/json"
        },
        body: JSON.stringify({
            url: "https://stripe.com/about",
            processing_type: "advanced"
        })
    }
);

const data = await response.json();
console.log(JSON.stringify(data, null, 2));
```

### Spanish video (transcription language)

**Request**

```json
{
  "processing_type": "advanced",
  "transcription_language": "es-US",
  "url": "https://example.com/videos/entrevista-consumidor.mp4"
}
```

**Response**

```json
{
  "job_id": "job_url_abc789",
  "status": "pending"
}
```

**SDK Code**

```python Python REST
import requests

BASE_URL = "https://api.filesearch.dev"
API_KEY = "your_api_key"

headers = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json"
}

# Spoken language is auto-detected by default; pass transcription_language
# only when you want to force it (e.g. mixed-language audio).
response = requests.post(
    f"{BASE_URL}/v2/collections/consumer_interviews/index/url",
    headers=headers,
    json={
        "url": "https://example.com/videos/entrevista-consumidor.mp4",
        "processing_type": "advanced",
        "transcription_language": "es-US"
    },
    timeout=60.0
)

if response.status_code in [200, 201]:
    data = response.json()
    print(f"Job started! ID: {data['job_id']}")
else:
    print(f"Error: {response.status_code}")
```

```typescript TypeScript REST
const BASE_URL = "https://api.filesearch.dev";
const API_KEY = "your_api_key";

// Spoken language is auto-detected by default; pass transcription_language
// only when you want to force it (e.g. mixed-language audio).
const response = await fetch(
    `${BASE_URL}/v2/collections/consumer_interviews/index/url`,
    {
        method: "POST",
        headers: {
            "Authorization": `Bearer ${API_KEY}`,
            "Content-Type": "application/json"
        },
        body: JSON.stringify({
            url: "https://example.com/videos/entrevista-consumidor.mp4",
            processing_type: "advanced",
            transcription_language: "es-US"
        })
    }
);

if (response.ok) {
    const data = await response.json();
    console.log(`Job started! ID: ${data.job_id}`);
} else {
    console.error(`Error: ${response.status}`);
}
```