AI You Don't Integrate

Multi-provider LLM support with vision, reasoning, and streaming. Anthropic, OpenAI, Gemini—one API, all providers.

On this page

Every AI product needs LLM integration. Without SystemPrompt, you're writing provider SDKs, handling rate limits, managing API keys, implementing streaming, and building fallback logic. That's months of work before your AI does anything useful.

The Problem

Integrating LLMs into production is harder than it looks:

  • Multiple providers: Anthropic, OpenAI, Google—each with different APIs, auth, and quirks
  • Capabilities vary: Vision works differently. Streaming varies. Tool calling isn't standard.
  • Cost tracking: Which requests cost what? Who's spending your API budget?
  • Error handling: Rate limits, timeouts, model deprecation, region availability
  • Request logging: What did the AI say? When? To whom? Why?

Most teams pick one provider and hope for the best. Then they need another provider and start over.

The Solution

SystemPrompt provides unified LLM execution across providers:

Multi-Provider Support

One configuration, all providers:

# services/ai/providers.yaml
providers:
  anthropic:
    api_key: ${ANTHROPIC_API_KEY}
    default_model: claude-sonnet-4-20250514
  openai:
    api_key: ${OPENAI_API_KEY}
    default_model: gpt-4o
  google:
    api_key: ${GOOGLE_API_KEY}
    default_model: gemini-2.0-flash

Unified Request API

Same interface regardless of provider:

# Execute with default provider
systemprompt admin agents message assistant "Analyze this data"

# Execute with specific provider
systemprompt admin agents message assistant "Analyze this data" --provider openai

# Execute with specific model
systemprompt admin agents message assistant "Analyze this data" --model claude-opus-4-20250514

Vision & Image Analysis

All vision-capable models work the same way:

# services/agents/image-analyst.yaml
name: image-analyst
model: claude-sonnet-4-20250514
capabilities:
  - vision
system_prompt: |
  You analyze images and provide detailed descriptions.

Supports:

  • Anthropic Claude: claude-sonnet-4, claude-opus-4 with vision
  • OpenAI: gpt-4o, gpt-4-vision
  • Google Gemini: gemini-2.0-flash, gemini-pro-vision

Extended Thinking & Reasoning

OpenAI o1 models with configurable reasoning:

# services/agents/researcher.yaml
name: researcher
model: o1
provider: openai
reasoning:
  effort: high  # low, medium, high
system_prompt: |
  You are a research assistant that thinks deeply about problems.

Streaming Responses

Real-time streaming for all providers:

# Stream response to terminal
systemprompt admin agents message assistant "Write a blog post" --stream

# Stream via API
curl -N https://yourdomain.com/api/v1/agents/assistant/stream \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"message": "Write a blog post"}'

Image Generation

Generate images with supported providers:

# services/agents/designer.yaml
name: designer
model: gemini-2.0-flash
capabilities:
  - image_generation

systemprompt admin agents message designer "Create a logo for a tech startup"

Google Search Grounding

Real-time search integration with Gemini:

# services/agents/researcher.yaml
name: researcher
model: gemini-2.0-flash
provider: google
capabilities:
  - search_grounding

The model can search the web and cite sources in responses.

Cost Tracking & Analytics

Every request is logged with full cost breakdown:

# View AI costs
systemprompt analytics ai costs --period month

# Breakdown by model
systemprompt analytics ai costs --by model

# Breakdown by agent
systemprompt analytics ai costs --by agent

Database schema tracks:

  • Input/output tokens
  • Cost per request
  • Model used
  • Latency
  • Cache hits (prompt caching)
-- ai_requests table
id, agent_id, user_id, provider, model,
input_tokens, output_tokens, cost_usd,
latency_ms, cache_hit, created_at

Request Logging

Full audit trail of all AI interactions:

# View recent AI requests
systemprompt infra logs --type ai.request

# Filter by agent
systemprompt infra logs --type ai.request --agent assistant

# View specific request details
systemprompt analytics ai requests show <request-id>

Why This Matters

Provider Independence

Don't lock yourself into one provider:

  • Start with Claude for quality
  • Add GPT-4 for specific use cases
  • Use Gemini for search grounding
  • Switch models without code changes

Production-Ready

Features you'd have to build yourself:

Without SystemPrompt With SystemPrompt
SDK integration per provider One unified API
Rate limit handling Built in
Cost tracking code Automatic
Request logging Automatic
Streaming implementation Works out of box
Vision handling per provider Unified interface
Error retry logic Configurable

Cost Control

Know exactly what your AI costs:

  • Per-request cost tracking
  • Per-user cost attribution
  • Per-agent cost breakdown
  • Budget alerts and limits

Supported Models

Anthropic

  • claude-opus-4-20250514 - Most capable
  • claude-sonnet-4-20250514 - Balanced
  • claude-haiku-3-20240307 - Fast & cheap

OpenAI

  • gpt-4o - Flagship multimodal
  • gpt-4-turbo - Fast GPT-4
  • o1 - Reasoning model
  • o1-mini - Fast reasoning

Google Gemini

  • gemini-2.0-flash - Fast multimodal
  • gemini-pro - Balanced
  • gemini-pro-vision - Vision specialist

Getting Started

Configure your providers:

# services/ai/providers.yaml
providers:
  anthropic:
    api_key: ${ANTHROPIC_API_KEY}
    default_model: claude-sonnet-4-20250514

  openai:
    api_key: ${OPENAI_API_KEY}
    default_model: gpt-4o

Create an agent:

# services/agents/assistant.yaml
name: assistant
model: claude-sonnet-4-20250514
system_prompt: |
  You are a helpful assistant.

Execute:

systemprompt admin agents message assistant "Hello, world!"

See the AI Reference for detailed configuration options.