Image moderation

Check a user-uploaded photo before it goes live — unsafe content, spam and whether it matches its caption — and block or escalate it, with the Image Moderation template.

Goal: people upload photos with their listings. Dcision looks at each photo before it goes live: unsafe images and spam are blocked, unrelated ones and photos that don't match their caption go to a moderator.

Create the decision

Templates → Image moderation → Create decision:

PartContent
Statephoto (image, required), caption (string)
unsafeprobability — nudity, violence, weapons, drugs or hateful symbols; minimum confidence 0.6
contentchoice — product, spam, unrelated, other
matches_captionprobability — the photo shows what the caption describes
Policiesunsafe ≥ 0.5 → block; content = spam → block; content = unrelated → escalate; matches_caption < 0.3 → escalate; fallback action escalate

photo is an image field: PNG, JPEG, GIF or WebP. Run this decision on OpenAI Decisions for the engine to see the image itself; on Jev or Laya it reads a description written by OpenAI.

Call it

curl -X POST https://api.dcision.io/v1/decisions/image-moderation \
  -H "Authorization: Bearer $DCISION_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "state": {
      "photo": {"url":"https://example.com/listing-photo.jpg","filename":"listing-photo.jpg"},
      "caption": "Used road bike, size M"
    }
  }'
200 OK (abridged)
{
  "schema": "image-moderation",
  "result": { "unsafe": 0.02, "content": "product", "matches_caption": 0.93 },
  "action": "continue",
  "action_reason": { "type": "default" },
  "metrics": { "engine": "openai", "input_files": 1 }
}

Act on it

const decision = await dcisionDecide("image-moderation", { photo: fileFromBytes(upload, upload.name), caption });
if (decision.action === "continue") return publish(listing);
if (decision.action === "block") return reject(listing, decision.action_reason);
return queueForModeration(listing, decision);

Tune it

  • Describe your platform in context: what a normal photo looks like there.
  • Add options to content for what you don't allow (watermarks, photos of screens).
  • Raise the minimum confidence of unsafe for public listings; unsure cases go to a moderator.

On this page