AI

On This Page

How A Chat Turn Flows

A turn goes: composer → useChat → client.chat.send → model stream → conversation UI.

The kit does not point useChat at a plain fetch URL. Instead, a custom transport.sendMessages calls the oRPC streaming procedure client.chat.send, then converts the result with eventIteratorToUnproxiedDataStream from @orpc/client. On the server, streamToEventIterator(result.toUIMessageStream({ sendReasoning: true, sendSources: true })) turns the model stream into UI message parts (including reasoning and sources).

Persistence is split on purpose:

1. The user message is saved before the model stream starts.
2. The assistant message and an AiUsageEvent row are written in onFinish, once tokens stop.

Package Layout

What's Wired In

The AI SDK And Gateway

Every model call goes through one adapter: gateway(modelId). Change providers by changing a string. One Vercel key (AI_GATEWAY_API_KEY) covers many model ids and one billing surface.

Default model. CHAT_DEFAULT_MODEL_ID lives in packages/shared/src/schemas/chat.ts. The picker list lives in apps/web/components/dashboard/ai-chat/chat-view/models.ts (each entry carries display metadata). To show a model in the picker, add it there. The server already accepts any string gateway() understands.

Env. Chat needs one AI-specific variable: AI_GATEWAY_API_KEY. It is listed in turbo.json for cache invalidation, but not yet in apps/web/.env.example.

Supported providers and current model ids: models.dev. Any id the gateway accepts there works in gateway(modelId) here.

The Streaming Procedure

chat.send (POST /rpc/chat/send) runs five checks before it opens a model connection:

After the gates pass, the user Message is persisted, then:

const result = streamText({
  model: gateway(modelId),
  system: "You are a helpful assistant.",
  messages: modelMessages,
  onError: async () => {
    // Refund the reservation row so a failed turn does not consume the cap.
  },
  onFinish: async ({ text }) => {
    // Persist the assistant Message and (on the first turn)
    // generateText({ model: gateway("openai/gpt-4o-mini") }) to set
    // Chat.title and flip Chat.titleGenerated. The AiUsageEvent row has
    // already been written by reserveAiUsageEvent before streaming began.
  },
});

return streamToEventIterator(
  result.toUIMessageStream({
    sendReasoning: true,
    sendSources: true,
  })
);

chat.regenerate uses the same gates and streamText shape. In onFinish it deletes the trailing assistant turn(s) before inserting the replacement, and writes AiUsageEvent with kind: "chat_regenerate" instead of "chat_send".

The Client: useChat With An oRPC Transport

useChat from @ai-sdk/react drives the UI. The kit bridges it to oRPC with one transport helper: call client.chat.send, then convert the event iterator into the data stream useChat expects.

import { useChat } from "@ai-sdk/react";
import { eventIteratorToUnproxiedDataStream } from "@orpc/client";
import { client } from "@/lib/orpc";

const { messages, sendMessage, status, stop } = useChat({
  id: chatId,
  messages: seedMessages,
  transport: {
    async sendMessages(options) {
      const latestMessage = options.messages[options.messages.length - 1];
      return eventIteratorToUnproxiedDataStream(
        await client.chat.send(
          {
            chatId: options.chatId,
            messages: buildSendPayloadMessages(latestMessage),
            model: webSearchRef.current ? undefined : modelRef.current,
            webSearch: webSearchRef.current,
          },
          { signal: options.abortSignal }
        )
      );
    },
    reconnectToStream() {
      throw new Error("Unsupported");
    },
  },
  onFinish: () => {
    // Invalidate the sidebar list and dashboard stats.
  },
});

Patterns used around that bridge:

AI Elements: Building Blocks

Reusable presentation primitives, grouped by job. Import from @syntaxkit/ui/components/ai-elements/<file>.

Billing And Limits

Two surfaces gate AI usage: plan features (what you may do) and per-request limits (what fits in one payload).

Per-plan features

The cap counts AiUsageEvent rows (kind: "chat_send" or "chat_regenerate") in the active billing window. See Billing for plan config, and Storage: How An Upload Flows for the attachment pipeline.

Per-request limits

Defined in packages/shared/src/schemas/chat.ts and enforced by Zod on the server.

History is trimmed newest-first in buildModelContextMessages, so long threads keep the recent tail.

Abuse Protection

chat.send and chat.regenerate are surfaces in the shared abuse policy. Each request is keyed by userId and organizationId. Both must be present; otherwise the surface fails closed.

Tune limits in packages/shared/src/abuse.ts. Each surface declares its window and limit per characteristic, so you can lower the per-user cap without changing the per-org cap.

Adding A New AI Feature

Use the same shape as chat: schema, optional quota, procedure, return type, client, UI. Works for streaming or one-shot features (summarize, alt text, draft email, and similar).

export const summarizeInputSchema = z.object({
  text: z.string().min(1).max(50_000),
  style: z.enum(["bullet", "tldr", "executive"]).default("tldr"),
});

import { CHAT_DEFAULT_MODEL_ID } from "@syntaxkit/shared";

export const summarize = authorized
  .use(withActiveOrganization)
  .route({ path: "/summarize", method: "POST" })
  .input(summarizeInputSchema)
  .handler(async ({ context, input }) => {
    const billing = await getBillingState(context.organization.id);
    const usage = await reserveAiUsageEvent(billing, context.organization.id, {
      kind: "chat_send",
      chatId: null,
      createdByUserId: context.user.id,
    });
    try {
      const result = streamText({
        model: gateway(CHAT_DEFAULT_MODEL_ID),
        system: "Summarize the input in the requested style.",
        prompt: input.text,
        onError: () =>
          prisma.aiUsageEvent
            .delete({ where: { id: usage.id } })
            .catch(() => {}),
      });
      return streamToEventIterator(result.toUIMessageStream());
    } catch (error) {
      await prisma.aiUsageEvent
        .delete({ where: { id: usage.id } })
        .catch(() => {});
      throw error;
    }
  });

Where To Go Next

Also useful: Storage for the attachment pipeline, Database for the AI models, and Customization for the shadcn / Tailwind layer the ai-elements primitives extend.