Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions sdk/ATTACHMENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Objective
I want the AI to be able to handle uploading attachments. Things such as upload images and PDFs, etc.

The user should be able to upload the attachment by clicking on the "+" sign in sdk/widget/src/components/Composer.tsx and selectting on the file. They should also be able to drag and drop a file from their computer into the chat widget.

Upon uploading the file, the Composer should show a very small attachment preview of what they've attached.

Upon hitting submit with the attachment, the attachments, along with the message, should be sent to Vercel AI-SDK on the backend.

# Consider
Also consider how we can handle images on the backend too. I want to be able to do cool workflows like ask the AI "What is in this image" and it be able to tell me what it is by analyzing the image.
11 changes: 10 additions & 1 deletion sdk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,13 @@ The next step is to set up your AI agent backend. Create an API endpoint with yo

Here's a simple AI SDK UI stream agent:
```tsx
import {
createOpenAI,
convertWidgetMessagesToModelMessages,
streamText,
type UIMessage,
} from "@openchatwidget/sdk";

app.use(express.json());
app.post("/api/chat", async (request, response) => {
const { messages } = request.body as { messages: UIMessage[] };
Expand All @@ -62,13 +69,15 @@ app.post("/api/chat", async (request, response) => {
const result = streamText({
model: openai("gpt-5-mini"),
system: "You are the OpenChatWidget example assistant. Keep answers concise and useful.",
messages: await convertToModelMessages(messages),
messages: await convertWidgetMessagesToModelMessages(messages),
});

result.pipeUIMessageStreamToResponse(response);
});
```

`convertWidgetMessagesToModelMessages` normalizes uploaded file parts (including browser `data:` URLs) so server implementors do not need to manually preprocess attachments.

### 3. Connect the widget to the agent.

Grab the exact URL of your agent endpoint and paste it into `<YOUR_AGENT_STREAMING_ENDPOINT>`, for example `http://localhost:8787/api/chat`. Make sure to start both your front end and Node backend. You should be able to start chatting.
Expand Down
Loading
Loading