Insights You Don't Build
Full observability for AI operations. Costs, trends, usage, audit trails. Every request tracked automatically.
On this page
AI operations generate massive amounts of data. Who's using what? What's it costing? Is something broken? Without observability, you're flying blind. SystemPrompt tracks everything automatically.
The Problem
AI observability requires:
- Cost tracking: Which models cost what? Who's spending your budget?
- Usage analytics: What agents are popular? Which tools are used most?
- Audit logging: What happened, when, by whom?
- Error monitoring: What's failing? Why?
- Trend analysis: Is usage growing? Are costs increasing?
- Fraud detection: Is someone abusing the system?
Building this infrastructure takes months. Then you discover you're not tracking what matters.
The Solution
SystemPrompt provides comprehensive AI analytics out of the box.
Cost Tracking
Every AI request is tracked with full cost breakdown:
# View total AI costs
systemprompt analytics ai costs
# Costs by time period
systemprompt analytics ai costs --period month
# Costs by model
systemprompt analytics ai costs --by model
# Costs by agent
systemprompt analytics ai costs --by agent
# Costs by user
systemprompt analytics ai costs --by user
Output:
AI Costs (Last 30 Days)
-----------------------
Total: $1,247.32
By Model:
claude-sonnet-4 $892.15 (71.5%)
gpt-4o $312.44 (25.0%)
gemini-2.0-flash $42.73 (3.4%)
By Agent:
content-writer $543.21 (43.5%)
researcher $412.88 (33.1%)
assistant $291.23 (23.3%)
Usage Analytics
Track what's being used:
# Platform overview
systemprompt analytics overview
# Agent usage
systemprompt analytics agents usage
# Tool usage
systemprompt analytics mcp calls --period week
# Content engagement
systemprompt analytics content stats
Audit Logging
Complete audit trail of all operations:
# View recent logs
systemprompt infra logs
# Filter by event type
systemprompt infra logs --type ai.request
systemprompt infra logs --type agent.invoke
systemprompt infra logs --type mcp.tool_call
systemprompt infra logs --type auth.login
# Filter by user
systemprompt infra logs --user user_123
# Filter by agent
systemprompt infra logs --agent content-writer
# Filter by time
systemprompt infra logs --since "2026-01-01" --until "2026-01-28"
# Follow logs in real-time
systemprompt infra logs --follow
Event Categories
All events are categorized:
AI Events
ai.request - LLM request initiated
ai.response - LLM response received
ai.error - LLM error occurred
ai.tool_call - Function/tool invoked
Agent Events
agent.message - Message sent to agent
agent.response - Agent responded
agent.invoke - Agent invoked another agent
agent.error - Agent error
MCP Events
mcp.tool_call - MCP tool invoked
mcp.tool_result - Tool returned result
mcp.server_start - MCP server started
mcp.server_error - MCP server error
Auth Events
auth.login - User logged in
auth.logout - User logged out
auth.failed - Auth attempt failed
auth.token_issued - Token created
Permission Events
permission.check - Permission verified
permission.denied - Access denied
permission.granted - Access granted
Structured Log Format
Logs are structured for querying:
{
"timestamp": "2026-01-28T10:30:00Z",
"event_type": "ai.request",
"trace_id": "trace_abc123",
"user_id": "user_456",
"tenant_id": "tenant_789",
"agent_id": "content-writer",
"provider": "anthropic",
"model": "claude-sonnet-4-20250514",
"input_tokens": 1523,
"output_tokens": 892,
"cost_usd": 0.0234,
"latency_ms": 1847,
"status": "success"
}
User Analytics
Track user behavior:
# Top users by usage
systemprompt analytics users top --period month
# User activity trends
systemprompt analytics users trends
# User engagement
systemprompt analytics users engagement
Traffic Analysis
Understand traffic patterns:
# Traffic overview
systemprompt analytics traffic overview
# Geographic breakdown
systemprompt analytics traffic geo
# Device/browser breakdown
systemprompt analytics traffic devices
# Referrer analysis
systemprompt analytics traffic referrers
Fraud Detection
Automatic anomaly detection:
# View suspicious activity
systemprompt analytics security anomalies
# Fingerprint reputation
systemprompt analytics security fingerprints
# Bot traffic detection
systemprompt analytics traffic bots
Tracks:
- Unusual request patterns
- Geographic anomalies
- Device fingerprint reputation
- Rate limit violations
- Authentication failures
Custom Events
Track custom engagement events:
# Track custom event
systemprompt analytics events track "feature_used" \
--property feature=ai_chat \
--property duration=300
# In agent response, track events
tracking:
events:
- type: task_completed
properties:
task_type: research
duration_seconds: 45
Dashboards & Reports
Generate reports:
# Daily summary
systemprompt analytics report daily
# Weekly report
systemprompt analytics report weekly --email admin@example.com
# Export data
systemprompt analytics export --format csv --period month > analytics.csv
Database Schema
Analytics data is stored in dedicated tables:
Tables:
├── ai_requests - All LLM requests with costs
├── ai_request_tool_calls - Tool calls within AI requests
├── engagement_events - Custom tracking events
├── sessions - User sessions
├── fingerprint_reputation - Fraud signals
├── funnel_progress - Conversion tracking
└── analytics_* views - Aggregated analytics
Why This Matters
Cost Control
Know exactly what AI costs:
| Without SystemPrompt | With SystemPrompt |
|---|---|
| Surprise API bills | Real-time cost tracking |
| No user attribution | Per-user cost breakdown |
| Manual calculation | Automatic aggregation |
| No alerts | Budget alerts |
Compliance
Audit trails for:
- SOC 2: Access logging, change tracking
- GDPR: Processing records, data access
- HIPAA: Audit trails, access controls
- Enterprise: Complete activity history
Operations
Identify issues before users complain:
# Error rate monitoring
systemprompt analytics errors summary --period hour
# Latency tracking
systemprompt analytics performance latency
# Model availability
systemprompt analytics ai availability
Business Intelligence
Understand your AI product:
- Which features are used most?
- Which users are most engaged?
- What content performs best?
- Where do users drop off?
Getting Started
Analytics is enabled by default. View your data:
# Overview
systemprompt analytics overview
# AI costs
systemprompt analytics ai costs
# Recent activity
systemprompt infra logs --limit 50
Configure retention:
# services/analytics/config.yaml
analytics:
retention:
logs: 90d
events: 365d
aggregates: forever
export:
enabled: true
destination: s3://your-bucket/analytics
See the Analytics Reference for detailed configuration options.