Skip to main content
Section navigation

Files Service

Three-layer file storage with local and S3 backends, permission-scoped access, upload handling, and CDN-ready serving for AI workloads.

On this page

TL;DR -- systemprompt.io ships a file storage service that works out of the box with local disk and scales to S3-compatible cloud storage. Every operation passes through three layers: a unified service API, a permission check scoped to tenants and owners, and a swappable storage provider. Public files are served CDN-ready. Private files require authentication or a signed URL.

What It Does and Why It Matters

The files service handles upload, storage, retrieval, and access control for every file in a systemprompt.io deployment. Static assets, user uploads, AI-generated artifacts, and internal platform files all flow through the same interface.

This matters because AI workloads produce and consume files constantly. Agents generate reports, store code artifacts, and retrieve documents across sessions. Without a unified file layer, each of these operations would require separate plumbing for storage, permissions, and serving. The files service consolidates all of that into a single API surface with consistent behavior regardless of whether files live on local disk or in a cloud bucket.

Key capabilities:

  • Unified API -- upload(), download(), list(), and delete() work identically across all storage backends.
  • Tenant isolation -- Every file is scoped to a tenant. No cross-tenant access is possible.
  • Permission-scoped access -- Files have owners, visibility levels, and access scopes enforced on every operation.
  • CDN-ready serving -- Public files are served at stable paths suitable for caching and CDN distribution.
  • Signed URLs -- Temporary access to private files without requiring API authentication.

Three-Layer Architecture

The file storage system is built on three layers that separate concerns cleanly.

API Request / CLI Command / MCP Tool
         │
         ▼
┌─────────────────────────┐
│   Service Layer         │  Unified API: upload, download, list, delete
│   (systemprompt-files)  │  Content-type detection, metadata tracking
└────────┬────────────────┘
         │
         ▼
┌─────────────────────────┐
│   Permission Layer      │  Tenant boundary enforcement
│                         │  Owner + visibility checks
│                         │  Authentication validation
└────────┬────────────────┘
         │
         ▼
┌─────────────────────────┐
│   Provider Layer        │  Local filesystem or S3-compatible cloud
│                         │  Swappable via configuration
└─────────────────────────┘

Service layer

The systemprompt-files crate exposes the public API. It handles content-type detection, checksum calculation, metadata storage in the database, and delegates actual byte storage to the configured provider. Every file gets a database record with a unique ID, tenant association, path, MIME type, size, and integrity hash.

Permission layer

Before any read or write reaches the provider, the permission layer validates access. It checks:

  • Tenant boundary -- the requesting user belongs to the file's tenant.
  • Ownership -- the user owns the file, or the file has appropriate visibility.
  • Visibility -- public files are accessible without authentication; private files require a valid token and matching tenant scope.

This layer integrates with the authentication system. API tokens, session cookies, and signed URLs all resolve through the same permission path.

Provider layer

Storage providers implement the physical read/write operations. The provider is selected by a single configuration value and can be changed without modifying application code.

Provider Use Case Configuration Key
local Development, single-server deployments files.provider: local
s3 Production, multi-server, cloud-native files.provider: s3

Both providers organize files identically, so switching between them requires only a configuration change and a data migration.

Local Storage

Local storage keeps files on the server filesystem. It is the default provider and requires no external dependencies.

# .systemprompt/profiles/local/profile.yaml
files:
  provider: local
  local:
    root: "./storage/files"
    max_file_size_mb: 100

Files are organized under the configured root directory by tenant and purpose:

storage/files/
├── tenants/
│   └── <tenant_id>/
│       ├── files/           # User uploads
│       └── artifacts/       # Agent-generated content
├── public/                  # Shared assets (images, CSS, JS)
└── system/                  # Platform files

Local storage is suitable for development and single-server deployments. For multi-server or high-availability setups, use the S3 provider.

Cloud Storage (S3)

The S3 provider stores files in any S3-compatible service: AWS S3, MinIO, DigitalOcean Spaces, Backblaze B2, or Cloudflare R2.

# .systemprompt/profiles/production/profile.yaml
files:
  provider: s3
  s3:
    bucket: "your-bucket-name"
    region: "us-east-1"
    endpoint: null          # Default AWS endpoint; set for S3-compatible services
    access_key_id: ${AWS_ACCESS_KEY_ID}
    secret_access_key: ${AWS_SECRET_ACCESS_KEY}

For S3-compatible services that use a custom endpoint:

files:
  provider: s3
  s3:
    bucket: "my-bucket"
    region: "auto"
    endpoint: "https://your-minio-server.example.com"
    access_key_id: ${MINIO_ACCESS_KEY}
    secret_access_key: ${MINIO_SECRET_KEY}

Files are stored with tenant-prefixed keys for isolation. The same directory structure used by local storage maps to S3 object key prefixes.

Permission-Scoped Access

Every file has two access properties: an owner and a visibility level.

Property Values Effect
owner_id User ID or system Determines who can modify or delete the file
visibility public or private Determines who can read the file

Public files are served without authentication at /files/<path>. They are suitable for static assets, public images, and downloadable resources. These paths are stable and cacheable, making them ready for CDN distribution.

Private files require authentication. The API validates the requestor's token and confirms they belong to the file's tenant before serving content.

# Access a private file via API
curl -H "Authorization: Bearer <token>" \
  https://your-domain.com/api/v1/files/<file_id>

Signed URLs provide temporary access to private files without requiring API authentication. They are useful for embedding private images in emails, sharing files with external users, or generating time-limited download links.

# Generate a signed URL (expires in 1 hour)
systemprompt core files sign <file_id> --expires 3600

# Output: https://your-domain.com/files/signed/<token>

Upload Handling

Files are uploaded through the API or CLI. Each upload creates a database record and stores content in the configured backend.

CLI upload:

# Upload a file
systemprompt core files upload /path/to/local/file.pdf

# Upload with explicit content type and visibility
systemprompt core files upload /path/to/image.png \
  --content-type "image/png" \
  --visibility public

# Upload to a specific directory path
systemprompt core files upload /path/to/doc.pdf \
  --destination "/documents/reports/2026/"

Image processing:

Images can be resized and optimized during upload. Thumbnail generation is available for preview purposes.

# Upload image with processing
systemprompt core files upload photo.jpg \
  --process resize:800x600 \
  --process optimize

File metadata stored on upload:

Property Description
id Unique file identifier (UUID)
tenant_id Owning tenant
owner_id User who uploaded the file
path Logical file path within the tenant
filename Original filename
content_type Detected or specified MIME type
size File size in bytes
visibility public or private
checksum SHA-256 content hash for integrity verification
created_at Upload timestamp

CDN-Ready Serving

Public files are served at stable, predictable URLs that work with CDN caching and reverse proxies.

Endpoint Description
/files/<path> Public file serving (cacheable, no auth required)
/api/v1/files/<id> Authenticated API file access
/api/v1/files/<id>/download Force-download with Content-Disposition header
/files/signed/<token> Signed URL access (temporary, no auth required)

Public file URLs follow the pattern /files/<logical-path>, which maps directly to the storage layout. This makes it straightforward to place a CDN in front of the file serving endpoint and set cache headers appropriately.

For production deployments, configure your reverse proxy or CDN to cache responses from /files/ with long TTLs. Files are immutable once uploaded -- updating a file creates a new version at a new path.

File Operations for AI Agents

Agents interact with files through MCP tools, using the same permission model as the API. This enables workflows where agents generate content, store artifacts, and retrieve previous work across sessions.

Available agent operations:

  • Upload -- store generated reports, code, images, and other artifacts.
  • Download -- retrieve files for processing or analysis.
  • List -- browse available files within the agent's authorized scope.
  • Search -- find files by name, path pattern, or metadata.

Artifacts produced by agents are stored under the artifacts/ directory within the tenant, with metadata linking them to the originating conversation and agent.

Storage Quotas

Tenants can have storage quotas to manage resource usage and prevent unbounded growth.

# Tenant settings (via cloud API)
tenant:
  settings:
    storage_quota_gb: 100
    max_file_size_mb: 50

# Check storage usage
systemprompt core files stats

# Example output:
# Total: 2.4 GB / 100 GB (2.4%)
# Files: 1,247
# Largest: document.pdf (156 MB)

Configuration Reference

Item Location Description
Provider Profile files.provider Storage backend: local or s3
Local root Profile files.local.root Directory for local file storage
Max file size Profile files.local.max_file_size_mb Per-file upload size limit (local)
S3 bucket Profile files.s3.bucket Cloud storage bucket name
S3 region Profile files.s3.region AWS region or auto for S3-compatible
S3 endpoint Profile files.s3.endpoint Custom endpoint for non-AWS S3 services
Storage quota Tenant settings storage_quota_gb Per-tenant total storage limit
File size limit Tenant settings max_file_size_mb Per-tenant upload size cap

CLI Reference

Command Description
systemprompt core files list List files with pagination and filtering
systemprompt core files show <id> Show detailed file information
systemprompt core files upload <path> Upload a file from the local filesystem
systemprompt core files delete <id> Delete a file by ID
systemprompt core files sign <id> Generate a signed URL for a private file
systemprompt core files validate <path> Validate a file before upload
systemprompt core files config Show file upload configuration
systemprompt core files search <pattern> Search files by path pattern
systemprompt core files stats Show file storage statistics
systemprompt core files ai AI-generated images operations

Run systemprompt core files <command> --help for detailed options on any command.