> ## Documentation Index
> Fetch the complete documentation index at: https://docs.omnifence.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Content moderation

> Automated safety screening for AI-generated media

Omnifence API includes a content moderation module that screens text prompts, chat turns, images, and
videos for unsafe content. Each modality is exposed as its own endpoint with a focused pipeline
tuned to that medium. Audio moderation is in development.

## Why moderation

AI generation platforms need to prevent harmful content from being created or distributed. The
moderation module automates this by analysing inputs against the configured
[content categories](/moderation/categories) and returning an `is_prohibited`
[verdict](/moderation/decisions).

## Modalities

### Prompt moderation

Screen a **text prompt** against your blocklist and an LLM prohibition check before generation.

* **Input**: `text` field
* **Pipeline**: a blocklist scan followed by an LLM prohibition check (your account administrator can
  enable additional optional steps)
* **Output**: `is_prohibited` verdict, plus a `reason` when a prompt is rejected

[Learn more about prompt moderation](/moderation/prompt-moderation)

### Chat moderation

Screen a **single chat turn** with the LLM prohibition check only. The blocklist is skipped, so
conversational text isn't rejected for incidental mentions of flagged terms.

* **Input**: `text` field
* **Pipeline**: an LLM prohibition check (no blocklist scan)
* **Output**: `is_prohibited` verdict, plus a `reason` when a turn is rejected

[Learn more about chat moderation](/moderation/chat-moderation)

### Image moderation

Screen a **single image** with vision-language and NSFW classifiers, plus any per-client custom
categories you have configured.

* **Input**: `image` field — a public HTTP/HTTPS URL (direct file upload is currently paused)
* **Pipeline**: VLM classification + NSFW classification + optional custom VLM children, all in
  parallel
* **Output**: `is_prohibited` verdict and flagged categories

[Learn more about image moderation](/moderation/image-moderation)

### Video moderation

Analyse a **video** by sending the whole clip to a video-capable vision model — the same checks as
image moderation, applied to the video directly.

* **Input**: `video` field — a public HTTP/HTTPS URL up to 300 MB (direct file upload is currently paused)
* **Pipeline**: the video is classified directly by the vision model (no frame extraction)
* **Output**: `is_prohibited` verdict and an informational `nsfw` label

A legacy frame-by-frame variant is also available at
[`POST /moderate/video/frame`](/api-reference/endpoint/video-frame-moderate): it samples frames with
FFmpeg, classifies each one, and merges the results (any prohibited frame rejects the video).

[Learn more about video moderation](/moderation/video-moderation)

### Coming soon

* [Audio moderation](/moderation/audio-moderation)

## Asynchronous processing

All modalities run asynchronously. When you submit a request, the API returns a `job_id` immediately
with `status: "queued"`. You can then:

1. **Poll** the [job status endpoint](/api-reference/endpoint/get-job) for results.
2. **Stream** real-time updates via [SSE](/platform/sse-progress).
3. **Receive** results via [webhook](/platform/webhooks) when the job completes.

## Pipeline architecture

Moderation jobs use a parent-child architecture. The parent job coordinates one or more child jobs,
each handling a specific check. All children must complete before the parent aggregates the results
into a final decision.

### Prompt moderation pipeline

```
PROMPT_MODERATION (parent)
└── TEXT_MODERATION          — Blocklist scan, LLM prohibition check
```

### Chat moderation pipeline

```
CHAT_MODERATION (parent)
└── CHAT_LLM_CHECK           — LLM prohibition check (no blocklist)
```

### Image moderation pipeline

```
IMAGE_MODERATION (parent)
├── VLM_CLASSIFICATION       — Visual analysis via LLM (one per enabled VLM category)
├── NSFW_CLASSIFICATION       — GPU-based NSFW detection
└── VLM_CLASSIFICATION        — Per-client custom categories (one per enabled custom category)
```

### Video moderation pipeline

```
VIDEO_MODERATION (parent)
├── VLM_CLASSIFICATION       — Video analysis via vision model (AI Adult General, always on)
└── VLM_CLASSIFICATION       — NSFW label (when enabled)
```

### Video frame moderation pipeline (legacy)

```
VIDEO_FRAME_MODERATION (parent)
└── FRAME_EXTRACTION         — Sample frames from video via FFmpeg
    └── FRAME_MODERATION     — Per-frame content analysis (one per frame)
```

If any child job fails, the parent job fails.