Twelve thousand product images. No descriptions. No categories. No color tags. Just filenames like IMG_4892.jpg and product_0371_front.png sitting in an S3 bucket, completely invisible to search engines and useless for filtering. That was the state of LingerieMuseum.com's product catalog when I inherited it. The site had thousands of pages that Google could not understand because every product had a photo and a price, but no machine-readable description of what was actually in the image.
The obvious solution was to use a cloud vision API - Google Vision, Amazon Rekognition, or GPT-4V - to classify and describe every image. I got quotes. Google Vision would cost roughly $18,000 for the initial batch plus ongoing costs for new products. GPT-4V would be even more. For a niche e-commerce site that generates modest revenue, that math does not work. So I built a private, on-premise image recognition pipeline that processed all 12,000 images for the cost of electricity. Here is exactly how it works, what went wrong along the way, and why I would do it again without hesitation.
The Problem: 12,000 Invisible Products
LingerieMuseum.com is a fashion product catalog - think of it as a curated database of intimate apparel products with images, pricing, and retail links. The data came from various product feeds, and while the feeds included basic category information, it was inconsistent and sparse. About 60% of products had no category at all. 80% had no color information. 95% had no fabric or material description. None had the kind of rich, structured product attributes that make a catalog searchable and SEO-friendly.
What I needed for each product:
- Primary category - bra, underwear, sleepwear, swimwear, loungewear, etc.
- Subcategory - push-up bra, boyshort, silk chemise, bikini top, etc.
- Colors - specific color names, not just "multi"
- Fabric/material - lace, silk, cotton, satin, mesh, etc.
- Style tags - everyday, romantic, sporty, luxe, minimalist, etc.
- SEO description - 2-3 sentence product description for the page
Doing this manually would take one person roughly 6-8 months of full-time work at 80-100 products per day. At $20/hour, that is $25,000-$33,000 in labor. Not happening.
Cloud API Cost Analysis: Why I Said No
Before building anything custom, I priced out every major cloud vision and multimodal API. Here is what 12,000 images would cost for the initial classification pass:
| Service | Price Per Image | 12,000 Images | Ongoing (500/mo) | Annual Total |
|---|---|---|---|---|
| Google Cloud Vision API | $1.50/1K images | $18.00 | $0.75/mo | $27.00 |
| Amazon Rekognition | $1.00/1K images | $12.00 | $0.50/mo | $18.00 |
| GPT-4V (classification only) | ~$0.04/image | $480.00 | $20/mo | $720.00 |
| GPT-4V (full description) | ~$0.07/image | $840.00 | $35/mo | $1,260.00 |
| Imagga | $0.001-0.01/image | $12-120 | $0.50-5/mo | $18-180 |
| Clarifai | ~$0.0014/image | $16.80 | $0.70/mo | $25.20 |
Wait - Google and Amazon look cheap, right? Yes, but those prices are for basic label detection. "Clothing." "Person." "Blue." That is not what I need. I need structured product attributes in a specific schema. For that, you need a multimodal LLM like GPT-4V, and suddenly you are looking at $840 for the initial batch plus $35/month ongoing. Plus, the basic vision APIs return generic labels. They will tell you an image contains "clothing" and "blue," but they will not tell you it is a "push-up bra in midnight blue lace with adjustable straps." For that level of structured output, you need either GPT-4V or a fine-tuned model.
More importantly, this is not a one-time cost. New products arrive weekly. We add 400-600 new products per month. That ongoing API cost compounds. And I am fundamentally uncomfortable sending 12,000 product images - some of which feature models in intimate apparel - to third-party APIs where I have zero control over data retention, model training policies, or access logging.
The Solution: LLaVA on Local Hardware
LLaVA (Large Language and Vision Assistant) is an open-source multimodal model that combines a vision encoder (CLIP ViT-L/14) with a language model (Llama or Mistral). The 13B parameter version runs comfortably on a consumer GPU with 8GB VRAM when quantized to 4-bit precision. It accepts an image and a text prompt, and generates text responses - exactly what I need for structured product classification.
Hardware
I used the same GPU server that handles all our AI workloads at InnovaTek Solutions:
- NVIDIA RTX 3060 Ti (8GB VRAM)
- 64GB system RAM
- 2TB NVMe SSD
- Ubuntu 22.04 LTS
Total hardware cost allocated to this project: effectively $0, since the server was already running other workloads. If you are building from scratch, expect to spend $1,000-$1,500 for a capable inference server.
Software Stack
# Install Ollama (manages model lifecycle)
curl -fsSL https://ollama.com/install.sh | sh
# Pull LLaVA 1.6 13B quantized
ollama pull llava:13b-v1.6-q4_K_M
# Verify it loaded correctly
ollama list
# NAME SIZE
# llava:13b-v1.6-q4_K_M 8.0 GB
# Quick test with a sample image
ollama run llava:13b-v1.6-q4_K_M "Describe this product image in detail" --images ./test_product.jpgThe Classification Pipeline: Architecture and Code
The pipeline is straightforward: read images from disk, send each to LLaVA via Ollama's HTTP API with a structured prompt, parse the JSON response, and store results in PostgreSQL. I wrapped it in a Python script that runs as a systemd service for the initial batch and as a cron job for ongoing new products.
#!/usr/bin/env python3
"""
Product image classifier using LLaVA via Ollama.
Processes images in batches, stores structured attributes in PostgreSQL.
"""
import os
import json
import base64
import time
import logging
import requests
import psycopg2
from psycopg2.extras import execute_values
from pathlib import Path
logging.basicConfig(level=logging.INFO, format='%(asctime)s %(levelname)s %(message)s')
logger = logging.getLogger(__name__)
OLLAMA_URL = "http://localhost:11434/api/generate"
MODEL = "llava:13b-v1.6-q4_K_M"
CLASSIFICATION_PROMPT = """Analyze this fashion/lingerie product image. Return a JSON object with these exact fields:
{
"category": "main category (bra, underwear, sleepwear, swimwear, loungewear, hosiery, accessory)",
"subcategory": "specific type (e.g., push-up bra, thong, silk chemise, bikini top)",
"colors": ["list", "of", "colors"],
"fabric": "primary material (lace, silk, cotton, satin, mesh, polyester, nylon)",
"style_tags": ["list", "of", "style", "descriptors"],
"description": "2-3 sentence product description suitable for an e-commerce page"
}
Return ONLY the JSON object. No explanation, no markdown formatting."""
def classify_image(image_path: str, max_retries: int = 3) -> dict | None:
"""Send image to LLaVA and parse structured response."""
with open(image_path, "rb") as f:
image_b64 = base64.b64encode(f.read()).decode()
for attempt in range(max_retries):
try:
resp = requests.post(OLLAMA_URL, json={
"model": MODEL,
"prompt": CLASSIFICATION_PROMPT,
"images": [image_b64],
"stream": False,
"options": {
"temperature": 0.1,
"num_predict": 512,
"top_p": 0.9,
}
}, timeout=120)
raw = resp.json()["response"]
# Strip markdown code fences if present
raw = raw.strip().removeprefix("```json").removeprefix("```").removesuffix("```").strip()
return json.loads(raw)
except (json.JSONDecodeError, KeyError) as e:
logger.warning(f"Attempt {attempt+1} failed for {image_path}: {e}")
if attempt < max_retries - 1:
time.sleep(2)
continue
except requests.exceptions.Timeout:
logger.error(f"Timeout processing {image_path}")
return None
logger.error(f"All retries failed for {image_path}")
return None
def process_batch(image_dir: str, db_conn, batch_size: int = 50):
"""Process all unclassified images in directory."""
cursor = db_conn.cursor()
# Get already-processed filenames
cursor.execute("SELECT filename FROM product_classifications")
processed = {row[0] for row in cursor.fetchall()}
# Find unprocessed images
image_files = [
f for f in Path(image_dir).glob("*")
if f.suffix.lower() in ('.jpg', '.jpeg', '.png', '.webp')
and f.name not in processed
]
logger.info(f"Found {len(image_files)} unprocessed images")
results = []
for i, img_path in enumerate(image_files):
classification = classify_image(str(img_path))
if classification:
results.append((
img_path.name,
classification.get("category", "unknown"),
classification.get("subcategory", ""),
json.dumps(classification.get("colors", [])),
classification.get("fabric", ""),
json.dumps(classification.get("style_tags", [])),
classification.get("description", ""),
))
if len(results) >= batch_size:
_flush_to_db(cursor, results)
db_conn.commit()
results = []
if (i + 1) % 100 == 0:
logger.info(f"Processed {i+1}/{len(image_files)} images")
if results:
_flush_to_db(cursor, results)
db_conn.commit()
def _flush_to_db(cursor, results):
"""Bulk insert classification results."""
execute_values(cursor, """
INSERT INTO product_classifications
(filename, category, subcategory, colors, fabric, style_tags, description)
VALUES %s
ON CONFLICT (filename) DO UPDATE SET
category = EXCLUDED.category,
subcategory = EXCLUDED.subcategory,
colors = EXCLUDED.colors,
fabric = EXCLUDED.fabric,
style_tags = EXCLUDED.style_tags,
description = EXCLUDED.description,
updated_at = NOW()
""", results)
logger.info(f"Flushed {len(results)} results to database")
if __name__ == "__main__":
conn = psycopg2.connect(
host="localhost", dbname="lingeriemuseum",
user=os.environ["DB_USER"], password=os.environ["DB_PASS"]
)
process_batch("/data/product-images", conn)
conn.close()PostgreSQL Schema
CREATE TABLE product_classifications (
id SERIAL PRIMARY KEY,
filename VARCHAR(255) UNIQUE NOT NULL,
category VARCHAR(50) NOT NULL DEFAULT 'unknown',
subcategory VARCHAR(100),
colors JSONB DEFAULT '[]',
fabric VARCHAR(100),
style_tags JSONB DEFAULT '[]',
description TEXT,
confidence_score FLOAT,
created_at TIMESTAMP DEFAULT NOW(),
updated_at TIMESTAMP DEFAULT NOW()
);
CREATE INDEX idx_classifications_category ON product_classifications(category);
CREATE INDEX idx_classifications_colors ON product_classifications USING GIN(colors);
CREATE INDEX idx_classifications_tags ON product_classifications USING GIN(style_tags);Cron Job for Ongoing Processing
# /etc/cron.d/product-classifier
# Run every night at 2 AM to catch new product images
0 2 * * * appuser cd /opt/classifier && \
DB_USER=classifier DB_PASS=$(cat /run/secrets/db_pass) \
python3 classify_products.py >> /var/log/classifier.log 2>&1Performance: Real Numbers From the 12K Batch
The initial batch of 12,000 images took roughly 60 hours of GPU time spread over 5 days. Here are the actual performance metrics:
| Metric | Value |
|---|---|
| Total images processed | 12,247 |
| Successful classifications | 11,893 (97.1%) |
| Failed (bad image / parse error) | 354 (2.9%) |
| Average time per image | 17.6 seconds |
| Throughput | ~204 images/hour |
| GPU utilization during processing | 82-94% |
| GPU temperature (sustained) | 71-76C |
| Total electricity consumed | ~12 kWh ($1.68) |
Let me repeat that last number. $1.68 in electricity. For classifying 12,000 images with structured product attributes, color extraction, fabric detection, and SEO descriptions. Compare that to $840 on GPT-4V or even $18 on Google Vision (which would not have given us structured descriptions anyway).
Quality Assessment: How Accurate Is LLaVA?
I manually reviewed 500 randomly selected classifications against the actual product images. Here is the accuracy breakdown:
| Field | Accuracy | Notes |
|---|---|---|
| Category | 96.4% | Occasionally confused loungewear/sleepwear |
| Subcategory | 89.2% | Weaker on niche subcategories |
| Primary color | 93.8% | Struggles with dark navy vs black |
| Secondary colors | 81.4% | Sometimes misses subtle accent colors |
| Fabric/material | 78.6% | Hardest field - lace vs mesh is tricky in photos |
| Style tags | 91.0% | Generally good, occasionally too generic |
| Description quality | 87.4% | Usable as-is for ~87% of products |
The 78.6% accuracy on fabric detection was the weakest link. In product photos, the difference between lace and mesh is often invisible without very high-resolution macro shots. I solved this by cross-referencing against the product feed data - when the feed included fabric information (about 30% of products), I used that as ground truth and only relied on LLaVA for the remaining 70%.
Common Failure Modes
Understanding where the model fails is as important as knowing where it succeeds:
- All-black products on dark backgrounds - LLaVA could not distinguish product edges, leading to garbled descriptions. Fix: pre-process with background removal using rembg.
- Multi-pack products - Images showing 3-pack or 5-pack product sets confused the model. It would describe one item and ignore the rest. Fix: add "this image shows a multi-pack product set" to the prompt when the filename contained "pack" or "set".
- Very small images (<200px) - Thumbnails produced unreliable results. Fix: skip images below 300px minimum dimension and flag for manual review.
- Watermarked images - Heavy watermarks occasionally got described as product features. Fix: no good automated fix, flagged for manual review.
# Pre-processing checks we added after the first batch
from PIL import Image
def preprocess_check(image_path: str) -> tuple[bool, str]:
"""Check if image is suitable for classification."""
try:
img = Image.open(image_path)
w, h = img.size
if min(w, h) < 300:
return False, "too_small"
if img.mode not in ('RGB', 'RGBA'):
img = img.convert('RGB')
img.save(image_path)
# Check if image is mostly black (dark product on dark bg)
if img.convert('L').getextrema()[1] < 40:
return False, "too_dark"
return True, "ok"
except Exception as e:
return False, f"corrupt: {e}"The Business Impact: What 12,000 Classifications Actually Changed
This was not just a technical exercise. The structured product data had a direct, measurable impact on the business:
SEO Results (3 Months Post-Implementation)
- Organic search traffic - Up 34% for product pages. Google could finally understand what was on each page.
- Indexed pages - 3,200 additional pages indexed (products that previously had thin content).
- Average position for product queries - Improved from 28.4 to 16.7 across target keywords.
- Rich snippet eligibility - Product pages now qualify for Google's product rich snippets with structured data.
User Experience
- Category filtering - Users can now filter by category, color, fabric, and style. Before, the only filter was price range.
- Search quality - On-site search using the generated descriptions returns relevant results instead of garbage.
- Time on site - Up 22% as users can actually find products they want.
Operational Efficiency
- New product processing - Automated. No more manual data entry for product attributes.
- Content team freed up - The part-time contractor who was manually writing product descriptions now focuses on editorial content instead.
What I Would Do Differently
If I were starting this project today, three things would change:
1. Use LLaVA 1.6 from the start. I initially used LLaVA 1.5, which had noticeably worse accuracy on subcategory and fabric detection. I re-ran about 4,000 images after upgrading to 1.6. Starting with the newer version would have saved 20 hours of GPU time.
2. Build the pre-processing pipeline first. I lost time on the first batch processing images that should have been filtered out - corrupt files, thumbnails, heavily watermarked images. Building the quality checks upfront would have reduced the 354 failures to maybe 50.
3. Use a two-pass approach for descriptions. The classification pass and the description generation pass have different optimal prompts. Classification works best with a terse, JSON-only prompt. Description generation benefits from more context about the brand, target audience, and SEO keywords. Combining them in one prompt leads to compromises on both. I eventually split them into two passes for new products, and the description quality improved significantly.
The Privacy Advantage Nobody Talks About
Here is something that matters more than most people realize: every one of those 12,000 product images features intimate apparel. Some are modeled. Sending those images to a cloud API means they traverse the public internet, get processed on shared infrastructure, and are subject to the API provider's data retention and training policies.
OpenAI's API terms say they do not use API data for training (as of March 2026). But they do log requests for abuse monitoring, and their data processing agreements have been updated three times in the past year. Google's Cloud Vision API processes images on shared infrastructure with retention policies that vary by region and tier.
With on-premise processing, the images never leave the server. There is no network transmission to intercept. There is no third-party data processing agreement to read and re-read every time the provider updates their terms. There is no risk that a data breach at the API provider exposes our product catalog. For fashion and intimate apparel, this is not a theoretical concern - it is a business risk that our on-premise setup eliminates entirely.
Replicating This for Your Business
This pipeline is not specific to fashion e-commerce. The same architecture works for any business that needs to classify, describe, or analyze images at scale:
- Real estate - Classify room types, detect features (pool, fireplace, renovated kitchen), generate listing descriptions
- Insurance - Analyze damage photos for claims processing
- Manufacturing - Visual quality inspection of products on assembly lines
- Retail - Automated product photography analysis and catalog enrichment
- Healthcare - Medical image preliminary classification (with appropriate regulatory compliance)
The total cost to replicate our setup: a $1,000-$1,500 GPU server, a weekend of setup time, and about 200 lines of Python. That is it. No API keys, no usage quotas, no monthly invoices, no data leaving your network. Just a GPU doing exactly what GPUs were designed to do.
Twelve thousand images. $1.68 in electricity. And a product catalog that went from invisible to searchable overnight. That is the real business case for private AI.
Leave a Comment