This tutorial provides a complete, production-ready Python script for generating descriptive captions for images using OpenAI's Vision models (like gpt-4o)
This tutorial provides a complete, production-ready Python script for generating descriptive captions for images using OpenAI's Vision models (like gpt-4o). It's designed for developers who need to automate image annotation tasks.
In today's digital world, large volumes of images are generated and managed across various platforms – e-commerce, content management systems, social media, digital asset management, and more. Manually writing descriptive captions for thousands or even millions of images is a time-consuming, costly, and often inconsistent process. This lack of proper metadata hinders searchability, accessibility (for visually impaired users), and overall content discoverability.
This script addresses the challenge of automating image captioning. By leveraging advanced AI vision models, it can rapidly process a directory of images and generate relevant, high-quality descriptions, significantly reducing manual effort and improving the utility and accessibility of image libraries.
The AI Image Caption Generator script operates by iterating through a specified directory of images, processing each one, and then using a powerful AI model to generate a textual description. The core architecture involves several key steps:
gpt-4o model (or chosen alternative) analyzes the visual content and returns a textual caption.Key design decisions include using argparse for a robust command-line interface, Pillow for efficient image manipulation, base64 encoding for API compatibility, and json for standardized output. Robust error handling is implemented at each stage, from file operations to API interactions, to ensure reliability in production environments.
#!/usr/bin/env python3
"""
AI Image Caption Generator
Generates descriptive captions for a directory of images using OpenAI's Vision API.
The captions are saved to a JSON file.
"""
import os
import argparse
import json
import base64
import io
from pathlib import Path
from typing import List, Dict, Any, Optional
from PIL import Image
from openai import OpenAI, OpenAIError
# --- Configuration ---
# Environment variable for OpenAI API key
OPENAI_API_KEY: Optional[str] = os.environ.get("OPENAI_API_KEY")
# Default OpenAI model to use for captioning (gpt-4o is recommended for vision)
DEFAULT_MODEL: str = "gpt-4o"
# Default output filename for the JSON captions
DEFAULT_OUTPUT_FILENAME: str = "captions.json"
# Maximum dimension (width or height) for images sent to OpenAI.
# Resizing helps reduce token usage and improves processing speed for very large images.
IMAGE_MAX_DIMENSION: int = 1024
# Maximum number of tokens for the AI's response (caption length)
MAX_TOKENS: int = 150
# Supported image file extensions
SUPPORTED_IMAGE_EXTENSIONS: List[str] = [".jpg", ".jpeg", ".png", ".webp"]
# --- Helper Functions ---
def validate_api_key(api_key: Optional[str]) -> str:
"""Validates and returns the OpenAI API key."""
if not api_key:
raise ValueError(
"OpenAI API key not found. Please set the OPENAI_API_KEY environment variable."
)
return api_key
def encode_image_to_base64(image_path: Path) -> str:
"""
Encodes an image to a base64 string after resizing it.
This reduces the data sent to the API, optimizing cost and speed.
"""
try:
with Image.open(image_path) as img:
# Resize image while maintaining aspect ratio
img.thumbnail((IMAGE_MAX_DIMENSION, IMAGE_MAX_DIMENSION), Image.LANCZOS)
buffered = io.BytesIO()
# Save as JPEG for consistent base64 encoding, quality 85
img.save(buffered, format="JPEG", quality=85)
return base64.b64encode(buffered.getvalue()).decode("utf-8")
except FileNotFoundError:
raise FileNotFoundError(f"Image file not found: {image_path}")
except Exception as e:
raise IOError(f"Error processing image {image_path}: {e}")
def generate_caption(client: OpenAI, base64_image: str, model: str) -> str:
"""
Generates a caption for a base64 encoded image using OpenAI's Vision API.
"""
try:
response = client.chat.completions.create(
model=model,
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "Describe this image in detail, focusing on key objects, actions, and the overall scene. Keep the caption concise but informative."},
{
"type": "image_url",
"image_url": {"url": f"data:image/jpeg;base64,{base64_image}"},
},
],
}
],
max_tokens=MAX_TOKENS,
)
caption = response.choices[0].message.content
if not caption:
raise ValueError("AI did not return a caption.")
return caption.strip()
except OpenAIError as e:
raise OpenAIError(f"OpenAI API error: {e}")
except Exception as e:
raise RuntimeError(f"An unexpected error occurred during caption generation: {e}")
# --- Main Logic ---
def main():
"""
Main function to parse arguments, process images, and generate captions.
"""
parser = argparse.ArgumentParser(
description="Generate AI captions for images in a specified directory."
)
parser.add_argument(
"--dir",
type=str,
required=True,
help="Path to the directory containing images.",
)
parser.add_argument(
"--model",
type=str,
default=DEFAULT_MODEL,
help=f"OpenAI model to use for captioning (default: {DEFAULT_MODEL}).",
)
parser.add_argument(
"--output",
type=str,
default=DEFAULT_OUTPUT_FILENAME,
help=f"Output JSON file name for captions (default: {DEFAULT_OUTPUT_FILENAME}).",
)
args = parser.parse_args()
# Validate OpenAI API key
try:
api_key = validate_api_key(OPENAI_API_KEY)
client = OpenAI(api_key=api_key)
except ValueError as e:
print(f"Error: {e}")
exit(1)
image_dir = Path(args.dir)
output_file = Path(args.output)
model_name = args.model
if not image_dir.is_dir():
print(f"Error: Directory '{image_dir}' not found or is not a directory.")
exit(1)
print(f"Processing images in: {image_dir}")
print(f"Using OpenAI model: {model_name}")
captions: List[Dict[str, str]] = []
processed_count = 0
skipped_count = 0
for file_path in image_dir.iterdir():
if file_path.is_file() and file_path.suffix.lower() in SUPPORTED_IMAGE_EXTENSIONS:
print(f" Processing image: {file_path.name}")
try:
base64_image = encode_image_to_base64(file_path)
caption = generate_caption(client, base64_image, model_name)
captions.append({"filename": file_path.name, "caption": caption})
print(f" Caption generated: {caption[:70]}...") # Print first 70 chars
processed_count += 1
except (IOError, OpenAIError, RuntimeError, ValueError) as e:
print(f" Error processing {file_path.name}: {e}")
skipped_count += 1
except Exception as e:
print(f" An unexpected error occurred for {file_path.name}: {e}")
skipped_count += 1
else:
print(f" Skipping non-image or unsupported file: {file_path.name}")
skipped_count += 1
try:
with open(output_file, "w", encoding="utf-8") as f:
json.dump(captions, f, indent=4, ensure_ascii=False)
print(f"\nSuccessfully generated captions for {processed_count} images.")
if skipped_count > 0:
print(f"Skipped {skipped_count} files due to errors or unsupported formats.")
print(f"Captions saved to: {output_file.resolve()}")
except IOError as e:
print(f"Error: Could not write captions to {output_file}: {e}")
exit(1)
if __name__ == "__main__":
main()
The script relies on a few key configuration parameters, primarily through an environment variable and command-line arguments:
OPENAI_API_KEY: This is a required environment variable. You must set your OpenAI API key before running the script.export OPENAI_API_KEY="your_api_key_here"set OPENAI_API_KEY="your_api_key_here"$env:OPENAI_API_KEY="your_api_key_here"--dir <path/to/images> (Command-line argument):--dir ./my_photos/--model <model_name> (Command-line argument):gpt-4o, which offers excellent vision capabilities. You could also try gpt-4o-mini for potentially lower cost and faster responses if its quality is sufficient for your needs.gpt-4o--model gpt-4o-mini--output <filename.json> (Command-line argument):captions.json--output my_image_descriptions.jsonIMAGE_MAX_DIMENSION (Constant in script):1024 pixels.MAX_TOKENS (Constant in script):150 tokens.First, ensure you have the required dependencies installed:
pip install openai Pillow
Create an images directory with some sample images:
.
├── caption_gen.py
└── images
├── landscape.jpg
├── portrait.png
└── city_night.jpeg
1. Basic Usage (using default model and output filename):
Set your OpenAI API key (replace sk-... with your actual key):
export OPENAI_API_KEY="sk-YOUR_ACTUAL_OPENAI_API_KEY"
Run the script:
python3 caption_gen.py --dir ./images/
Expected Output on Console:
Processing images in: images
Using OpenAI model: gpt-4o
Processing image: landscape.jpg
Caption generated: A serene landscape features rolling green hills under a bright sky with light...
Processing image: portrait.png
Caption generated: A close-up portrait shows a person with short, dark hair and thoughtful eyes...
Processing image: city_night.jpeg
Caption generated: A vibrant night cityscape showcases tall buildings illuminated against a dark...
Successfully generated captions for 3 images.
Captions saved to: /path/to/your/project/captions.json
Expected captions.json content:
[
{
"filename": "landscape.jpg",
"caption": "A serene landscape features rolling green hills under a bright sky with light clouds, a winding road in the foreground, and distant mountains."
},
{
"filename": "portrait.png",
"caption": "A close-up portrait shows a person with short, dark hair and thoughtful eyes, set against a softly blurred background."
},
{
"filename": "city_night.jpeg",
"caption": "A vibrant night cityscape showcases tall buildings illuminated against a dark sky, with blurred lights from traffic on a busy street below, creating a dynamic urban scene."
}
]
2. Specifying a different model and output filename:
python3 caption_gen.py --dir ./images/ --model gpt-4o-mini --output my_custom_captions.json
Expected Output on Console:
Similar to the basic usage, but indicating gpt-4o-mini as the model and my_custom_captions.json as the output file.
3. Handling a directory with mixed files (unsupported types):
If you have other files like document.pdf or text.txt in the ./images/ directory:
python3 caption_gen.py --dir ./images/
Expected Output on Console (example with an unsupported file):
Processing images in: images
Using OpenAI model: gpt-4o
Processing image: landscape.jpg
Caption generated: A serene landscape features rolling green hills under a bright sky with light...
Skipping non-image or unsupported file: document.pdf
Processing image: portrait.png
Caption generated: A close-up portrait shows a person with short, dark hair and thoughtful eyes...
Successfully generated captions for 2 images.
Skipped 1 files due to errors or unsupported formats.
Captions saved to: /path/to/your/project/captions.json
The script is designed with robust error handling to gracefully manage various issues that may arise during execution:
OPENAI_API_KEY environment variable is not set, the script will immediately exit with an informative error message.--dir does not exist or is not a directory, the script will print an error and exit.jpg, jpeg, png, and webp extensions. Any other file types in the directory will be skipped with a message indicating they are unsupported.Pillow errors during resizing or encoding will be caught. The specific image will be skipped, an error message printed to the console, and the script will continue processing other images.openai client will raise an OpenAIError.OpenAIError will be raised.OpenAIError.OpenAIError, prints a detailed error message for the specific image, and continues processing subsequent images.IOError will be caught, and an error message will be displayed.In general, the script aims to process as many images as possible, reporting errors for individual failures rather than halting the entire operation. A summary at the end indicates how many images were successfully processed and how many were skipped.
concurrent.futures.ThreadPoolExecutor or asyncio with aiohttp to send multiple API requests in parallel, significantly speeding up the caption generation. Be mindful of OpenAI's rate limits.generate_caption function to support different AI vision services (e.g., Google Cloud Vision API, Azure AI Vision, AWS Rekognition) or even local open-source models (e.g., LLaVA, BLIP) if performance and accuracy requirements allow, giving users more flexibility and potentially reducing costs.