AI

SyntaxKit ships a complete org-scoped streaming chat at /dashboard/ai-chat on top of the Vercel AI SDK and the Vercel AI Gateway. Everything routes through one adapter, gateway(modelId), so swapping providers is a string change rather than a refactor. The @syntaxkit/ui/components/ai-elements/* package gives you the same composer, transcript, attachments, reasoning, sources, model picker, and mic button to drop into any new AI feature.

We pick the AI Gateway over per-provider SDKs because it removes the "N keys for N providers" problem: one Vercel key, dozens of model ids, one billing surface, one place to switch when a new model lands.

How A Chat Turn Flows

The unusual bit is the transport. The kit pairs @ai-sdk/react's useChat with a custom transport.sendMessages that calls the oRPC streaming procedure client.chat.send and pipes the result through eventIteratorToUnproxiedDataStream from @orpc/client. The server returns streamToEventIterator(result.toUIMessageStream({ sendReasoning: true, sendSources: true })) so reasoning and sources arrive as first-class UI message parts. Persistence happens in onFinish so the user message is saved before the stream and the assistant message + an AiUsageEvent row land once tokens stop flowing.

The diagram source lives at apps/docs/diagrams/ai-chat-flow.mmd. Rerun pnpm --filter @syntaxkit/docs diagrams:build after editing it to refresh both SVG variants.

Package Layout

What's Wired In

The AI SDK And Gateway

Two design choices to call out before wiring:

The default model is CHAT_DEFAULT_MODEL_ID in packages/shared/src/schemas/chat.ts. The selectable model list lives in apps/web/components/dashboard/ai-chat/chat-view/models.ts (the models array) so each entry can carry display metadata (chef, provider slug for the logo, label). Adding a model is a two-line edit when you also want it in the picker; it's a zero-line edit when you just want the server to accept it (any string is forwarded to gateway()).

Env vars. The chat path needs exactly one AI-specific env: AI_GATEWAY_API_KEY (recognized by the AI SDK's gateway() adapter). The kit declares it in turbo.json for cache invalidation, but does not yet ship it in apps/web/.env.example.

The full list of supported providers and current model ids lives at models.dev; whatever string the gateway accepts there will work in gateway(modelId) here.

The Streaming Procedure

chat.send is a single-purpose oRPC procedure (POST /rpc/chat/send) that flows through five gates before it ever opens a model connection. Listed in order:

After the gates pass, the user Message is persisted, then:

const result = streamText({
  model: gateway(modelId),
  system: "You are a helpful assistant.",
  messages: modelMessages,
  onError: async () => {
    // Refund the reservation row so a failed turn does not consume the cap.
  },
  onFinish: async ({ text }) => {
    // Persist the assistant Message and (on the first turn)
    // generateText({ model: gateway("openai/gpt-4o-mini") }) to set
    // Chat.title and flip Chat.titleGenerated. The AiUsageEvent row has
    // already been written by reserveAiUsageEvent before streaming began.
  },
});

return streamToEventIterator(
  result.toUIMessageStream({
    sendReasoning: true,
    sendSources: true,
  })
);

chat.regenerate mirrors this: same gates, same streamText shape, but onFinish deletes the trailing assistant turn(s) before inserting the replacement and writes an AiUsageEvent with kind: "chat_regenerate" instead of "chat_send".

The Client: useChat With An oRPC Transport

The kit uses @ai-sdk/react's useChat over an oRPC streaming procedure rather than a plain fetch endpoint. The bridge is one helper:

import { useChat } from "@ai-sdk/react";
import { eventIteratorToUnproxiedDataStream } from "@orpc/client";
import { client } from "@/lib/orpc";

const { messages, sendMessage, status, stop } = useChat({
  id: chatId,
  messages: seedMessages,
  transport: {
    async sendMessages(options) {
      const latestMessage = options.messages[options.messages.length - 1];
      return eventIteratorToUnproxiedDataStream(
        await client.chat.send(
          {
            chatId: options.chatId,
            messages: buildSendPayloadMessages(latestMessage),
            model: webSearchRef.current ? undefined : modelRef.current,
            webSearch: webSearchRef.current,
          },
          { signal: options.abortSignal }
        )
      );
    },
    reconnectToStream() {
      throw new Error("Unsupported");
    },
  },
  onFinish: () => {
    // Invalidate the sidebar list and dashboard stats.
  },
});

A few patterns worth knowing:

AI Elements: Building Blocks

Reusable presentation primitives. Group them by what they do, not what file they live in. All export from @syntaxkit/ui/components/ai-elements/<file>.

Transcript Primitives

Composer Primitives

Streaming-Aware Primitives

Billing And Limits

Two limit surfaces gate AI usage. Per-plan features control what a user is allowed to do; per-request limits control what fits in a single payload.

Per-plan features

The cap is enforced by counting AiUsageEvent rows (kind: "chat_send" or "chat_regenerate") in the active billing window. See Billing for how plans and entitlements are configured, and Storage: How An Upload Flows for the attachment pipeline that feeds chat images.

Per-request limits

All defined in packages/shared/src/schemas/chat.ts and enforced by Zod on the server.

History is trimmed newest-first inside buildModelContextMessages, so an active conversation always keeps its tail and the trim shows up only on long threads.

Abuse Protection

chat.send and chat.regenerate are surfaces in the shared abuse policy. Each request is keyed by userId and organizationId; both characteristics must be present, otherwise the surface fails closed.

Tuning happens in packages/shared/src/abuse.ts: each surface declares its window and limit per characteristic, so you can lower the per-user cap without changing the per-org cap.

Adding A New AI Feature

Reuse the same handler shape for a new feature (summarize a doc, generate alt text, draft an email, anything). The pattern is the streaming chat in miniature.

export const summarizeInputSchema = z.object({
  text: z.string().min(1).max(50_000),
  style: z.enum(["bullet", "tldr", "executive"]).default("tldr"),
});

import { CHAT_DEFAULT_MODEL_ID } from "@syntaxkit/shared";

export const summarize = authorized
  .use(withActiveOrganization)
  .route({ path: "/summarize", method: "POST" })
  .input(summarizeInputSchema)
  .handler(async ({ context, input }) => {
    const billing = await getBillingState(context.organization.id);
    // Atomically reserve a slot in the monthly cap before spending
    // gateway tokens. Refund the reservation if the model errors.
    const usage = await reserveAiUsageEvent(billing, context.organization.id, {
      kind: "chat_send",
      chatId: null,
      createdByUserId: context.user.id,
    });
    try {
      const result = streamText({
        model: gateway(CHAT_DEFAULT_MODEL_ID),
        system: "Summarize the input in the requested style.",
        prompt: input.text,
        onError: () =>
          prisma.aiUsageEvent
            .delete({ where: { id: usage.id } })
            .catch(() => {}),
      });
      return streamToEventIterator(result.toUIMessageStream());
    } catch (error) {
      await prisma.aiUsageEvent
        .delete({ where: { id: usage.id } })
        .catch(() => {});
      throw error;
    }
  });

Where To Go Next