Items and Characteristics
Automatically check any videos and images for harmful content
The ITEMS_CHARACTERISTICS endpoints surface items and characteristics detected in an image (object recognition) or video (including sound). Each item or characteristic has a prevalence score which represents the extent to which it is present in a given piece of content. We can also analyse the text that is shared along the image or video.
You can use these scores in mappings or models to implement safety policies, for example setting a numerical threshold for automated flagging of content or in a linear regression model (read more in our guide for selecting thresholds).
The following 50+ items and characteristics are available:
knife- knives in all contexts, e.g. includes kitchen knives
violent_knife- knives in violent contexts, including hunting knives
alcohol- alcoholic drinks in bottles, cans or glasses
drink- drinks of any kinds
Visual content characteristics:
Audio language toxicity:
OCR language toxicity:
Caption language toxicity:
The classes below have been developed to beta standard. They can be included in offline evaluations and will be available as add-on features in early October 2023
🙋🏽♀️ Note: content containing these items and characteristics can still be detected with Custom policies and with our Brand Safety Framework. This is because these products use inputs that represent all aspects of the content - not just Items & Characteristics. For example, our GARM product flags harmful White Supremacy content as Hate Speech through learnt patterns in the content, even though this isn’t an Item/Characteristic.
medical- NSFW content in a medical context, e.g. partial nudity in a breast examination
- 1.Optical Character Recognition (OCR). OCR refers to any text that appears on the image or video. Examples include the captions for translations, words displayed on a T-shirt or hhandwritten content.
- 2.Speech audio transcriptions. A literal transcription of the speech detected during a video. This is only available in English. More languages are coming up soon, starting with Spanish. Please be aware that any sound in the video may interfere with the audio transcription.
You can find below an example of how to process OCR and audio transcriptions from the Items & Characteristics API response. They are included in the sections "ocr_texts" and "audio_texts".
"I'm very happy",
"I've been thinking about it and I'm happy"
Last modified 3h ago