Tracking & performance

Execution logging, database schema, Eloquent models, and analytics for prompt performance monitoring.

Introduction

Prompt Deck includes an optional database tracking system that logs prompt versions and execution data. This enables:

• Performance monitoring — Track token usage, latency, and cost per prompt version.
• A/B testing — Compare performance metrics across different prompt versions.
• Audit trails — Record what was sent to and received from AI providers.
• Cost analysis — Monitor API spending per prompt, model, and provider.
• User feedback — Attach ratings and comments to individual executions.

Tracking is entirely optional. Prompt Deck works fully without it.

Setup

Enable tracking

Set the tracking configuration in config/prompt-deck.php:

'tracking' => [
    'enabled'    => env('PROMPTDECK_TRACKING_ENABLED', true),
    'connection' => env('PROMPTDECK_DB_CONNECTION'),
],

Or via environment variable:

PROMPTDECK_TRACKING_ENABLED=true

By default, tracking is disabled when APP_DEBUG=true and enabled in production.

Publish migrations

Publish and run the migrations to create the required tables:

php artisan vendor:publish --tag=prompt-deck-migrations
php artisan migrate

Database connection

By default, tracking uses your application’s default database connection. To store tracking data on a separate database:

PROMPTDECK_DB_CONNECTION=analytics

The connection name must match a connection defined in config/database.php.

Database schema

prompt_versions table

Stores prompt version records and tracks which version is active.

Column	Type	Nullable	Description
id	bigint	No	Auto-incrementing primary key.
name	string	No	The prompt name (e.g. order-summary).
version	unsigned int	No	The version number (e.g. 1, 2, 3).
system_prompt	text	Yes	The system prompt content.
user_prompt	text	No	The user prompt content.
metadata	json	Yes	Version metadata as JSON.
is_active	boolean	No	Whether this version is the active version. Default: false.
created_at	timestamp	Yes	Creation timestamp.
updated_at	timestamp	Yes	Last update timestamp.

Indexes:

• Unique index on (name, version) — ensures no duplicate versions per prompt.
• Index on (name, is_active) — fast lookups for the active version.

prompt_executions table

Logs individual prompt executions with performance data.

Column	Type	Nullable	Description
id	bigint	No	Auto-incrementing primary key.
prompt_name	string	No	The prompt name.
prompt_version	unsigned int	No	The prompt version used.
input	json	Yes	The input sent to the AI provider.
output	text	Yes	The response received from the AI provider.
tokens	unsigned int	Yes	Total token usage (input + output).
latency_ms	unsigned int	Yes	Round-trip time in milliseconds.
cost	decimal(8,6)	Yes	Estimated cost in your currency.
model	string	Yes	The AI model used (e.g. gpt-4o).
provider	string	Yes	The AI provider (e.g. openai, anthropic).
feedback	json	Yes	User feedback (ratings, comments, etc.).
created_at	timestamp	Yes	Execution timestamp.

Indexes:

• Index on (prompt_name, prompt_version, created_at) — fast filtering by prompt and time range.

The prompt_executions table does not have an updated_at column. Records are insert-only.

Recording executions

Manual tracking

Use the PromptDeck::track() method to record an execution manually:

use Veeqtoh\PromptDeck\Facades\PromptDeck;

PromptDeck::track('order-summary', 2, [
    'input'    => ['message' => 'Summarise order #1234'],
    'output'   => 'Your order contains 3 items totalling $150.00...',
    'tokens'   => 150,
    'latency'  => 234.5,
    'cost'     => 0.002,
    'model'    => 'gpt-4o',
    'provider' => 'openai',
    'feedback' => ['rating' => 5, 'comment' => 'Accurate summary'],
]);

All fields in the data array are optional. You can track as little or as much as you need:

// Minimal tracking — just record that the prompt was used
PromptDeck::track('order-summary', 2, []);

// Track only tokens and cost
PromptDeck::track('order-summary', 2, [
    'tokens' => 150,
    'cost'   => 0.002,
]);

If tracking is disabled in configuration, the track() method is a safe no-op — you can call it without checking configuration first.

Automatic tracking via middleware

When using the Laravel AI SDK integration, the TrackPromptMiddleware handles tracking automatically. See AI SDK — Performance Tracking Middleware for setup details.

Tracked fields

Field	Type	Description
input	array|null	The input data sent to the AI provider. Stored as JSON.
output	string|null	The text response from the AI provider.
tokens	int|null	Total token count (input + output tokens).
latency	float|null	Round-trip time in milliseconds. Stored as latency_ms.
cost	float|null	Estimated cost. Stored with up to 6 decimal places.
model	string|null	AI model identifier (e.g. gpt-4o, claude-3-sonnet).
provider	string|null	AI provider name (e.g. openai, anthropic).
feedback	array|null	Arbitrary feedback data. Stored as JSON.

Eloquent models

Prompt Deck provides two Eloquent models for interacting with tracking data.

PromptVersion

Veeqtoh\PromptDeck\Models\PromptVersion

Represents a prompt version record in the prompt_versions table.

use Veeqtoh\PromptDeck\Models\PromptVersion;

// Find the active version
$active = PromptVersion::where('name', 'order-summary')
    ->where('is_active', true)
    ->first();

// Get all versions for a prompt
$versions = PromptVersion::where('name', 'order-summary')
    ->orderBy('version')
    ->get();

Fillable attributes: name, version, system_prompt, user_prompt, metadata, is_active

Casts:

Attribute	Cast
version	integer
is_active	boolean
metadata	array

PromptExecution

Veeqtoh\PromptDeck\Models\PromptExecution

Represents an execution record in the prompt_executions table.

use Veeqtoh\PromptDeck\Models\PromptExecution;

// Get recent executions
$recent = PromptExecution::where('prompt_name', 'order-summary')
    ->latest()
    ->limit(100)
    ->get();

// Get executions for a specific version
$v2Executions = PromptExecution::where('prompt_name', 'order-summary')
    ->where('prompt_version', 2)
    ->get();

Fillable attributes: prompt_name, prompt_version, input, output, tokens, latency_ms, cost, model, provider, feedback

Casts:

Attribute	Cast
prompt_version	integer
tokens	integer
latency_ms	integer
cost	decimal:6
input	array
feedback	array

PromptExecution sets UPDATED_AT = null since execution records are insert-only and never updated.

Factories

Both models include factories for testing.

PromptVersionFactory

use Veeqtoh\PromptDeck\Models\PromptVersion;

// Create a basic version
$version = PromptVersion::factory()->create();

// Create an active version
$version = PromptVersion::factory()->active()->create();

// Create with a specific name and version
$version = PromptVersion::factory()
    ->named('order-summary')
    ->version(2)
    ->active()
    ->create();

Available states:

Method	Description
active()	Mark as the active version (is_active = true).
version(int $v)	Set a specific version number.
named(string $name)	Set a specific prompt name.

PromptExecutionFactory

use Veeqtoh\PromptDeck\Models\PromptExecution;

// Create a basic execution
$execution = PromptExecution::factory()->create();

// Create with feedback
$execution = PromptExecution::factory()
    ->withFeedback(['rating' => 5, 'comment' => 'Great'])
    ->create();

// Create a minimal execution (only required fields)
$execution = PromptExecution::factory()->minimal()->create();

// Create for a specific prompt
$execution = PromptExecution::factory()
    ->forPrompt('order-summary', 2)
    ->create();

Available states:

Method	Description
withFeedback(array $feedback = [])	Include user feedback. Generates random feedback if no argument given.
minimal()	Set all optional fields to null.
forPrompt(string $name, int $version)	Set a specific prompt name and version.

Querying execution data

Basic queries

use Veeqtoh\PromptDeck\Models\PromptExecution;

// Total executions for a prompt
$count = PromptExecution::where('prompt_name', 'order-summary')->count();

// Most recent execution
$latest = PromptExecution::where('prompt_name', 'order-summary')
    ->latest()
    ->first();

// Executions in the last 24 hours
$recent = PromptExecution::where('prompt_name', 'order-summary')
    ->where('created_at', '>=', now()->subDay())
    ->get();

Performance analysis

// Average latency for a prompt version
$avgLatency = PromptExecution::where('prompt_name', 'order-summary')
    ->where('prompt_version', 2)
    ->avg('latency_ms');

// Average token usage
$avgTokens = PromptExecution::where('prompt_name', 'order-summary')
    ->where('prompt_version', 2)
    ->avg('tokens');

// 95th percentile latency (approximate)
$p95 = PromptExecution::where('prompt_name', 'order-summary')
    ->orderBy('latency_ms')
    ->limit(1)
    ->offset((int) (PromptExecution::where('prompt_name', 'order-summary')->count() * 0.95))
    ->value('latency_ms');

A/B testing

Compare metrics across prompt versions:

// Compare average latency between v1 and v2
$comparison = PromptExecution::where('prompt_name', 'order-summary')
    ->whereIn('prompt_version', [1, 2])
    ->groupBy('prompt_version')
    ->selectRaw('prompt_version, AVG(latency_ms) as avg_latency, AVG(tokens) as avg_tokens, COUNT(*) as executions')
    ->get();

// Compare feedback ratings
$ratings = PromptExecution::where('prompt_name', 'order-summary')
    ->whereNotNull('feedback')
    ->whereIn('prompt_version', [1, 2])
    ->groupBy('prompt_version')
    ->selectRaw("prompt_version, AVG(JSON_EXTRACT(feedback, '$.rating')) as avg_rating")
    ->get();

Cost analysis

// Total cost for a prompt
$totalCost = PromptExecution::where('prompt_name', 'order-summary')
    ->sum('cost');

// Cost by model
$costByModel = PromptExecution::where('prompt_name', 'order-summary')
    ->groupBy('model')
    ->selectRaw('model, SUM(cost) as total_cost, COUNT(*) as executions')
    ->get();

// Daily cost trend
$dailyCost = PromptExecution::where('prompt_name', 'order-summary')
    ->where('created_at', '>=', now()->subDays(30))
    ->groupByRaw('DATE(created_at)')
    ->selectRaw('DATE(created_at) as date, SUM(cost) as daily_cost')
    ->orderBy('date')
    ->get();

Version management via database

When tracking is enabled, version activation is managed through the prompt_versions table instead of metadata.json files. This provides a centralised, queryable record of version history.

The PromptDeck::activate() method and prompt:activate command automatically use the appropriate storage (database or file) based on your tracking configuration.

// Activate programmatically
PromptDeck::activate('order-summary', 2);

// Query active versions
$activeVersions = PromptVersion::where('is_active', true)->get();

// Version history
$history = PromptVersion::where('name', 'order-summary')
    ->orderBy('version')
    ->get()
    ->map(fn ($v) => [
        'version'  => $v->version,
        'active'   => $v->is_active,
        'metadata' => $v->metadata,
    ]);