Multimodal

Multimodal

Multimodality refers to a model's ability to understand and generate content using various input types—such as text, images, audio, and video. Multimodal models combine diverse data sources to interpret complex contexts, enabling more comprehensive and nuanced responses.

VisionImagesspeech

AudioEvalsResponsesspeech

Realtime Eval Guide

Gpt-image-1.5 Prompting Guide

Transcribing User Audio with a Separate Realtime Request

AudioResponsesspeech

Realtime Prompting Guide

Generate images with high input fidelity

Using Evals API on Image Inputs

Practical guide to data-intensive apps with the Realtime API

ImagesResponsesVision

Image Understanding with RAG

AudiospeechTiktoken

Context Summarization with Realtime API

Other15

ElatoAI - Realtime Speech AI Agents for ESP32 on Arduino

Comparing Speech-to-Text Methods with the OpenAI API

Agents SDKAudiospeech

Generate images with GPT Image

Processing and narrating a video with GPT-4.1-mini's visual capabilities and GPT-4o TTS API

ResponsesspeechVision

Building a Voice Assistant with the Agents SDK

AudioResponsesspeech

Multi-Language One-Way Translation with the Realtime API

Using GPT4 Vision with Function Calling

Optimizing Retrieval-Augmented Generation using GPT-4o Vision Modality

CompletionsVision

Vision Fine-tuning on GPT-4o for Visual Question Answering

CompletionsFine-tuningVision

How to parse PDF docs for RAG

EmbeddingsVision

How to combine GPT4o mini with RAG to create a clothing matchmaker app

EmbeddingsVision

Using GPT4o mini to tag and caption images

EmbeddingsVision

Introduction to GPT-4o and GPT-4o mini

CompletionsVisionWhisper

Data Extraction and Transformation in ELT Workflows using GPT-4o as an OCR Alternative

CompletionsVision

CLIP embeddings to improve multimodal RAG with GPT-4 Vision

EmbeddingsVision