Color Palette AI Integration Guide

{
  "name": "extract_color_palette",
  "description": "Extract dominant colors from an image and return a hex palette with percentage distribution.",
  "parameters": {
    "type": "object",
    "properties": {
      "input_base64": {
        "type": "string",
        "description": "Base64-encoded image input. Accepts a data URI or raw base64 for JPG, PNG, or WebP images."
      },
      "max_colors": {
        "type": "integer",
        "description": "Maximum number of dominant colors to return.",
        "minimum": 2,
        "maximum": 16,
        "default": 8
      },
      "include_tints_shades": {
        "type": "boolean",
        "description": "Whether to include generated tints and shades for each dominant color.",
        "default": false
      }
    },
    "required": ["input_base64"]
  }
}

# Task Constraints: Color Palette Extraction Module

When extracting colors from images for AI workflows, you MUST keep the output deterministic and browser-compatible.

## Core Prohibitions
- NEVER assume server-side image libraries such as sharp, jimp, or pillow are available.
- NEVER sample the full-resolution image when a downscaled canvas is enough.
- NEVER return vague color names only. Always return hex values and percentages.

## Standard Implementation
1. Normalize input_base64 by removing the data URI prefix when present.
2. Decode the image into a Blob and load it into an ImageBitmap or HTMLImageElement.
3. Draw the image to a small canvas, typically capped around 256px on the longest side, for stable performance.
4. Ignore mostly transparent pixels.
5. Prefer skipping near-white and near-black pixels first, then fall back to all pixels if nothing remains.
6. Use a deterministic clustering strategy such as median cut to produce dominant color boxes.
7. Return each dominant color as hex plus RGB values and a percentage estimate.

## Output Rules
- Sort colors by percentage descending.
- If include_tints_shades is true, generate lighter and darker steps from each hex color.
- Fail with a clear validation error if the image cannot be decoded.

async function main(args) {
  const {
    input_base64,
    max_colors = 8,
    include_tints_shades = false,
  } = args;

  if (!input_base64) {
    return { success: false, result: [], error: "input_base64 is required" };
  }

  try {
    const base64Data = input_base64.replace(/^data:[^;]+;base64,/, "");
    const binaryString = atob(base64Data);
    const bytes = Uint8Array.from(binaryString, (char) => char.charCodeAt(0));
    const blob = new Blob([bytes]);
    const image = await createImageBitmap(blob);

    const maxDim = 256;
    const scale = Math.min(maxDim / image.width, maxDim / image.height, 1);
    const width = Math.max(1, Math.round(image.width * scale));
    const height = Math.max(1, Math.round(image.height * scale));

    const canvas = new OffscreenCanvas(width, height);
    const ctx = canvas.getContext("2d", { willReadFrequently: true });
    if (!ctx) {
      throw new Error("Canvas context unavailable");
    }

    ctx.drawImage(image, 0, 0, width, height);
    const { data } = ctx.getImageData(0, 0, width, height);

    const pixels = [];
    for (let i = 0; i < data.length; i += 4) {
      const alpha = data[i + 3];
      if (alpha < 128) continue;

      const r = data[i];
      const g = data[i + 1];
      const b = data[i + 2];
      const luma = 0.299 * r + 0.587 * g + 0.114 * b;
      if (luma < 5 || luma > 250) continue;
      pixels.push({ r, g, b });
    }

    if (pixels.length === 0) {
      for (let i = 0; i < data.length; i += 4) {
        pixels.push({ r: data[i], g: data[i + 1], b: data[i + 2] });
      }
    }

    const boxes = medianCut(pixels, max_colors);
    const totalPixels = pixels.length;

    const result = boxes
      .map((box) => {
        const count = box.pixels.length;
        if (!count) return null;

        const avgR = Math.round(box.pixels.reduce((sum, pixel) => sum + pixel.r, 0) / count);
        const avgG = Math.round(box.pixels.reduce((sum, pixel) => sum + pixel.g, 0) / count);
        const avgB = Math.round(box.pixels.reduce((sum, pixel) => sum + pixel.b, 0) / count);
        const hex = rgbToHex(avgR, avgG, avgB);

        const item = {
          hex,
          r: avgR,
          g: avgG,
          b: avgB,
          percentage: Math.round((count / totalPixels) * 100),
        };

        if (!include_tints_shades) return item;
        return {
          ...item,
          palette: generatePalette(hex, 4),
        };
      })
      .filter(Boolean)
      .sort((a, b) => b.percentage - a.percentage);

    return { success: true, result };
  } catch (error) {
    return {
      success: false,
      result: [],
      error: "Palette extraction failed: " + (error instanceof Error ? error.message : String(error)),
    };
  }
}