AI Chat

Scaffold comes with an AI chat built on laravel/ai. Users have threads they can return to, and the assistant can call tools that act on the app on their behalf. The chat is available on the web at /chat and through the API at POST api/chat.

Requires

An API key for at least one provider. Anthropic is the default, with OpenAI as the fallback. An Exa key is optional and adds web search for providers without their own.

ANTHROPIC_API_KEY=
OPENAI_API_KEY=
EXA_API_KEY=

The whole chat (pages, nav item and API endpoints) can be turned off with CONVERSATION_ENABLED=false.

Config

config/conversation.php   // providers, tools, thread boundaries, context
config/ai.php             // provider keys

Providers and Models

The providers are failover lists of provider => model. The agent list runs the conversation itself; fast is for the cheap housekeeping calls, which are the thread relevance check and summaries. At runtime the agents move AI_DEFAULT_PROVIDER to the front and drop any provider without an API key.

'providers' => [
    'agent' => [
        'anthropic' => env('CONVERSATION_AGENT_MODEL', 'claude-sonnet-5-5'),
        'openai' => env('CONVERSATION_AGENT_FALLBACK_MODEL', 'gpt-6.1-sol'),
    ],
    'fast' => [
        'anthropic' => env('CONVERSATION_FAST_MODEL', 'claude-haiku-5-5'),
        'openai' => env('CONVERSATION_FAST_FALLBACK_MODEL', 'gpt-6-luna'),
    ],
],

How It Works

The web chat (ThreadsController and the ThreadChat Livewire component) and the API (Api\ChatController) both go through App\Services\ConversationService.

Receiving a message

receive() files the message into a thread using Services\Conversation\ThreadResolver.

  • The web chat is explicit: it either continues the thread on screen or asks for a new one.
  • A message with no thread (the API without a thread_id) is placed automatically. Past idle_minutes of silence a new thread starts. Inside continuation_seconds, or for replies of short_reply_words or fewer, it always continues. A message starting with one of the new_thread_words ("new: buy milk") closes the thread and starts fresh. Anything in between is put to a fast-model ThreadRelevance check.
  • If the relevance check fails for any reason the message stays in the thread, since losing context is a worse mistake than carrying a little extra.
  • One thread per user is open at a time; continuing an old one reopens it.

Responding

respond() runs the App\Ai\Agents\Assistant agent on everything unanswered and records the reply, with its tool calls, provider and usage in meta.

  • The history is the last context_messages turns plus the thread summary.
  • Summaries come from the ThreadSummarizer agent, via the SummarizeThread job.
  • A provider failure becomes an apology reply, not an exception.

Livewire sends in two round trips (send then respond), so no queue or broadcasting is needed for the web chat.

API

Both endpoints use Sanctum. See API for tokens.

GET   api/chat/threads   // the user's threads, most recent first (paginated)
POST  api/chat           // send a message, throttled to 30 a minute
message=What's on my profile?   // required, max 4000 characters
thread_id=12                    // optional, continue this thread
new=true                        // optional, start a new thread

The response has the assistant's reply in data and the thread it was filed in under thread.

Adding a Tool

Tools let the assistant act on the app for the user it is talking to.

  1. Extend App\Ai\Tools\AppTool.
  2. Put the business logic in a service and call it from run().
  3. Add the class to conversation.tools.

Tools are resolved from the container with the user and thread, so a tool can also take services in its constructor. AppTool turns validation errors and exceptions into text the model can act on, rather than failing the whole reply. Tools/Users/* are the examples; they reuse UserService and UserResource like the MCP tools do.

class GetMyProfile extends AppTool
{
    public function description(): Stringable|string
    {
        return 'Get the profile of the user you are talking to.';
    }

    protected function run(Request $request): string
    {
        // call a service and return text for the model
    }
}

Web search

Web search is added on top of the configured tools automatically. If the leading provider has hosted web search it is used, otherwise the Exa-backed SearchWeb tool is added when EXA_API_KEY is set.

Retrieval (RAG)

Retrieval goes in as a tool too. RecallThreads, a keyword search over thread summaries, is the current retrieval. To add more, add an embeddings or vector-store tool (laravel/ai Embeddings/Stores) to conversation.tools. No vector infrastructure ships with Scaffold yet.

Testing

Tests fake the agents and never call a provider.

Assistant::fake(['Hello!'])->preventStrayPrompts();

// actually runs the tool, then replies
Assistant::fake([new ToolCall('id', 'GetMyProfile', []), 'Here you go']);

ThreadRelevance::fake([['related' => false, 'reason' => 'New topic']]);
ThreadSummarizer::fake(['A summary']);

Views

resources/views/chat/index.blade.php wraps the chat in v-pre, because Vue mounts on #app and would otherwise compile the chat content as a template.