Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
072a006
Add settings window and route renderer by view param
shaunandrews Apr 1, 2026
b252a92
Add chrome color tokens with light/dark mode support
shaunandrews Apr 1, 2026
b68f5e9
Add three-panel layout with resizable panels and keyboard shortcuts
shaunandrews Apr 1, 2026
051f859
Add new UI components: sidebar, site details, browser, settings tabs
shaunandrews Apr 1, 2026
e72ae79
Add dev platform switcher and theme-aware window background
shaunandrews Apr 1, 2026
cc35475
Add NEW-UI.md documenting the redesign architecture
shaunandrews Apr 1, 2026
303a9a0
Replace sidebar with Tasks placeholder and real SiteMenu/RunningSites
shaunandrews Apr 1, 2026
bdfdb57
Update site menu to use design tokens and animated drag-and-drop
shaunandrews Apr 1, 2026
f36f71a
Persist panel sizes and collapsed state across sessions
shaunandrews Apr 1, 2026
7316180
Add Studio Components settings tab with mocked site data
shaunandrews Apr 1, 2026
19a7630
Simplify site details and toolbar for panel layout integration
shaunandrews Apr 1, 2026
1a4eba9
Wire SiteContentTabs into primary panel and tighten spacing
shaunandrews Apr 1, 2026
5dbe9fd
Update NEW-UI.md with sidebar, site menu, and panel persistence work
shaunandrews Apr 1, 2026
817b950
Migrate all settings into the new settings window
shaunandrews Apr 1, 2026
7eb1743
Remove old settings modal and showUserSettings IPC
shaunandrews Apr 1, 2026
c12b508
Remove DevController overlay in favor of settings Automattician tab
shaunandrews Apr 1, 2026
fb5d534
Update NEW-UI.md with settings window migration details
shaunandrews Apr 1, 2026
00af662
Move system prompt to tools/common/ai/ for sharing between CLI and de…
shaunandrews Apr 2, 2026
717985e
Add task data model, IPC handlers, and persistence foundation
shaunandrews Apr 2, 2026
390f82b
Add Redux tasks slice with IPC event listeners
shaunandrews Apr 2, 2026
a7d6195
Wire task list into sidebar and add panel switching
shaunandrews Apr 2, 2026
139caf8
Add agent integration and chat UI
shaunandrews Apr 2, 2026
9dcb488
Add NEW-AGENT.md documenting Tasks and AI agent architecture
shaunandrews Apr 2, 2026
a3d67b7
Fix duplicate agent responses in chat
shaunandrews Apr 2, 2026
4929fe9
Persist task chat history to localStorage
shaunandrews Apr 2, 2026
29209c7
Add post_blocks_read and post_block_update MCP tools
shaunandrews Apr 2, 2026
1f8eee9
Add task archiving UI with archive toggle and clear all
shaunandrews Apr 2, 2026
99cfd01
Update docs for chat persistence, block tools, and archiving UI
shaunandrews Apr 2, 2026
49b5d81
Match sites by directory name in addition to display name
shaunandrews Apr 2, 2026
0a9b3da
Add embedded browser panel with auto-login and iframe preview
shaunandrews Apr 2, 2026
2bef136
Redesign error fallback with centered layout and dot grid background
shaunandrews Apr 2, 2026
7d81c19
Update NEW-UI.md with browser panel and task chat details
shaunandrews Apr 2, 2026
281f64c
Redesign task creation flow and archived tasks UI
shaunandrews Apr 2, 2026
0d404ba
Remove theme screenshot preview, Customize section, and all supportin…
shaunandrews Apr 2, 2026
2782ba8
Add browser control tools for the AI agent
shaunandrews Apr 2, 2026
2d6a12a
Update docs for browser tools, task UI changes, and new file references
shaunandrews Apr 2, 2026
592d23b
Delegate AI agent execution to CLI instead of importing SDK directly
shaunandrews Apr 2, 2026
25baf9a
Add progressive blur to task chat toolbar and input footer
shaunandrews Apr 2, 2026
8350fff
Add browser tabs for multi-page browsing in the preview panel
shaunandrews Apr 2, 2026
8ca6cfa
Fix task chat progressive blur to use sticky element inside scroll co…
shaunandrews Apr 2, 2026
c2c3059
Redesign task UI with floating toolbar and collapsible sidebar
shaunandrews Apr 2, 2026
3b668fa
Fix browser reload to stay on current page and make toolbar draggable
shaunandrews Apr 2, 2026
d6997ea
Add agent browser_navigate and browser_reload tools
shaunandrews Apr 2, 2026
e64b53f
Add image attachment support for agent messages
shaunandrews Apr 2, 2026
14497de
Update browser panel docs for agent IPC tools and toolbar changes
shaunandrews Apr 2, 2026
22b05e3
Consolidate tabs: remove Assistant and Import/Export, add Publish to …
shaunandrews Apr 3, 2026
6eb52d9
Fix toolbar layout and add keyboard-only focus rings
shaunandrews Apr 3, 2026
29f61ea
Update tests for tab consolidation and new panel layout
shaunandrews Apr 3, 2026
49ad1cb
Add element selection from browser preview for AI agent context
shaunandrews Apr 3, 2026
45e2ae2
Add area screenshot capture for browser preview
shaunandrews Apr 3, 2026
7ba06f5
Wire element selector and screenshot capture into browser panel
shaunandrews Apr 3, 2026
efebcc3
Update browser panel docs for element selection and screenshot capture
shaunandrews Apr 3, 2026
c01ec5c
Add image resize to prevent oversized base64 from crashing the agent
shaunandrews Apr 3, 2026
e530ebb
Add AI-generated task titles and summaries via Haiku
shaunandrews Apr 3, 2026
41de9ca
Add message queue so users can type follow-ups while agent is busy
shaunandrews Apr 3, 2026
077deeb
Replace inline tool cards with compact activity indicator
shaunandrews Apr 3, 2026
b05cbf9
Wire image resize, screenshot capture, and queue into chat input
shaunandrews Apr 3, 2026
83c1bac
Add task preview card on sidebar hover
shaunandrews Apr 3, 2026
7380003
Move select/capture to end of browser toolbar and add URL polling
shaunandrews Apr 3, 2026
44130df
Update docs for image resize, activity indicator, and toolbar changes
shaunandrews Apr 3, 2026
96a1d4d
Implement new site creation flow with dedicated window and routing
shaunandrews Apr 3, 2026
87dea29
Remove dedicated add-site window in favor of main-window creation flow
shaunandrews Apr 6, 2026
d789865
Move project creation flow into main window as first Task
shaunandrews Apr 6, 2026
4ebd7f1
Add questionnaire system for structured agent questions
shaunandrews Apr 6, 2026
f6d5bd0
Add preview server for local HTML files in the browser panel
shaunandrews Apr 6, 2026
1ff85d6
Redesign overview shortcuts with app icons and inline task list
shaunandrews Apr 6, 2026
77c422a
Render assistant messages as Markdown instead of plain text
shaunandrews Apr 6, 2026
8fb633a
Filter task list to existing sites and show setup tasks ungrouped
shaunandrews Apr 6, 2026
2ce4434
Add floating tour component for post-creation UI onboarding
shaunandrews Apr 6, 2026
e238c5c
Update docs, system prompt, and site-spec skill for creation flow
shaunandrews Apr 6, 2026
79eb534
Combine sidebar tasks and projects into unified project list
shaunandrews Apr 6, 2026
a1543f3
Replace custom Tooltip with WordPress Tooltip wrapper
shaunandrews Apr 6, 2026
4ecaf6e
Add inline task renaming, auto-create for single site, and title gene…
shaunandrews Apr 6, 2026
449a9e0
Fix panel layout sync, editable toolbar title, and update creation docs
shaunandrews Apr 6, 2026
bd402b6
Add sidebar footer with gravatar and settings, remove cog from toolbar
shaunandrews Apr 7, 2026
d02b046
Adjust sidebar typography: semibold projects, light tasks, smaller ar…
shaunandrews Apr 7, 2026
fb87d1b
Redesign import step with icons, two-column grid, and additional sources
shaunandrews Apr 8, 2026
bce137c
Add Notes: unified floating input with persistent highlights and note…
shaunandrews Apr 8, 2026
8cc908a
Update browser panel docs for Add Notes workflow and note overlays
shaunandrews Apr 8, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
242 changes: 242 additions & 0 deletions NEW-AGENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,242 @@
# Tasks & AI Agent

This branch adds AI-powered Tasks to Studio. A Task is a chat session with a Claude agent, tied to a WordPress site. Users can create tasks from the sidebar or site overview, chat with the agent, and the agent can manage their site via MCP tools and file operations.

## Status

**Proof of Concept** — The full pipeline is functional: task creation, sidebar navigation, chat UI, agent integration via the Claude Agent SDK in the Electron main process, and desktop-native MCP tools (site_list, site_info, site_start, site_stop, wp_cli). Tasks persist across app restarts. Permission prompts for filesystem operations outside the site directory are wired but not yet polished.

## Architecture

### Agent Execution Model

The Claude Agent SDK (`@anthropic-ai/claude-agent-sdk`) runs directly in the Electron main process. The SDK internally spawns its own subprocess (a bundled Claude Code CLI), so there's no double-nesting of child processes. The desktop app passes `pathToClaudeCodeExecutable` explicitly because Vite bundling breaks the SDK's internal path resolution.

```
Renderer (React) Main Process (Node.js)
| |
|-- ipcApi.startTaskAgentHandler --->|-- query({ prompt, mcpServers, ... })
| | SDK spawns its own subprocess
|<-- 'task-message' events ----------|-- for await (msg of query) {
|<-- 'task-message' events ----------| sendIpcEventToRenderer('task-message', msg)
| | }
|-- ipcApi.interruptTaskHandler ---->|-- query.interrupt()
```

### Provider Resolution

Authentication mirrors the CLI's provider fallback chain (`provider-resolver.ts`):

1. **WordPress.com** — Uses the shared auth token from `readSharedConfig()`. Proxies through the wpcom AI gateway.
2. **Anthropic Claude auth** — Checks for local Claude Code authentication via `claude auth status`.
3. **Anthropic API key** — Direct API key (not yet wired to UI for input).

### MCP Tools (Desktop-Native)

The agent has access to 12 Studio tools via an MCP server (`tools.ts`), all using `SiteServer` directly (not the CLI daemon):

- **site_list** — Lists all sites with status, paths, URLs.
- **site_info** — Detailed info for a specific site (path, URL, credentials, PHP version).
- **site_start** / **site_stop** — Start or stop a site's server.
- **wp_cli** — Execute WP-CLI commands on a running site (plugin install, post create, etc.).
- **post_blocks_read** — List all Gutenberg blocks in a post/page with indices, types, attributes, and content previews. Uses WordPress's `parse_blocks()` via `wp eval`.
- **post_block_update** — Replace a specific block by index with new block markup. Uses `parse_blocks()` / `serialize_blocks()` / `wp_update_post()` via `wp eval`. Markup is base64-encoded to avoid escaping issues.
- **browser_navigate** — Navigate the site preview browser to a URL or path. Syncs the visible preview iframe.
- **browser_reload** — Reload the current page. Syncs the visible preview iframe.
- **browser_screenshot** — Take a PNG screenshot via a hidden BrowserWindow's `webContents.capturePage()`. Returns an MCP image content block.
- **browser_read_page** — Read page title, URL, text content, and a structural DOM outline (headings, links, forms, buttons).
- **browser_console** — Read recent console messages (log/warning/error) with optional clear.

The browser tools use a `BrowserInspector` singleton (`browser-inspector.ts`) that manages hidden `BrowserWindow` instances per site. See `NEW-BROWSER.md` for details on the architecture.

The agent also has Claude Code's built-in file tools (Read, Write, Edit, Glob, Grep, Bash) for direct file manipulation. The agent's `cwd` is set to the task's site path so file operations work relative to the site.

### Permission Model

Read-only tools (Read, Glob, Grep, MCP tools) are auto-approved. Write operations (Write, Edit, Bash) within the site directory or temp directories are auto-approved. Write operations outside trusted roots trigger a permission prompt:

1. Main process `canUseTool` callback creates a pending Promise
2. Sends `task-permission-request` IPC event to renderer
3. Renderer shows inline permission dialog (Allow once / Allow for session / Deny)
4. User's response resolves the Promise via `respondToPermissionRequestHandler` IPC

Session-level approvals are cached per tool name so the user isn't prompted repeatedly.

### Data Model

**TaskMetadata** (persisted in `appdata-v1.json` alongside site data):

| Field | Type | Description |
|-------|------|-------------|
| `id` | `string` | UUID |
| `siteId` | `string` | Associated site ID |
| `title` | `string` | Auto-generated from first message |
| `status` | `'in-progress' \| 'waiting' \| 'done'` | Current state |
| `archived` | `boolean` | Hidden from sidebar when true |
| `createdAt` | `number` | Timestamp |
| `updatedAt` | `number` | Timestamp |
| `sessionId` | `string?` | SDK session ID for resuming conversations |

**TaskMessage** (persisted to `localStorage` via Redux listener, survives app restarts):

| Field | Type | Description |
|-------|------|-------------|
| `id` | `string` | Message ID (from SDK UUID or generated) |
| `role` | `'user' \| 'assistant' \| 'tool' \| 'system'` | Message type |
| `content` | `string` | Text content |
| `toolName` | `string?` | Tool name for tool-type messages |
| `toolInput` | `unknown?` | Tool input parameters |
| `toolResult` | `string?` | Tool execution result |
| `images` | `ImageAttachment[]?` | Base64 image attachments (user messages) |
| `elements` | `ElementAttachment[]?` | Selected DOM elements (user messages) |
| `isStreaming` | `boolean?` | Currently being streamed |
| `isError` | `boolean?` | Error message |

### Navigation State

Task selection lives in Redux (`tasks-slice.ts`) via `selectedTaskId`. This is separate from site selection (which uses `useSiteDetails` context). The two are mutually exclusive in the UI:

- Clicking a task sets `selectedTaskId`, primary panel shows `TaskChatPanel`
- Clicking a site clears `selectedTaskId`, primary panel shows `SiteContentTabs`
- Site list items check `selectedTaskId` to suppress their highlight when a task is active

### System Prompt

The agent uses the full CLI system prompt from `tools/common/ai/system-prompt.ts` (shared between CLI and desktop). This includes detailed workflow steps, design guidelines, block content rules, and all tool descriptions. The desktop app appends site-specific context (site name, path, instructions to use wp_cli for content and check revisions).

### Message Serialization

SDK messages (`SDKMessage` union type) are converted to flat `TaskMessage` objects by `message-serializer.ts`. The serializer handles:

- `assistant` messages — extracts text content and tool use blocks
- `user` messages (synthetic) — extracts tool result content blocks with `tool_use_id` matching
- `result` messages — surfaces error messages only (successful results are already shown via the preceding `assistant` message)
- `tool_use_summary` — shows tool execution summaries
- `tool_progress` — shows tool running indicators

Tool results are merged back onto their invocation message in Redux by matching `tool_use_id`, so each tool call appears as a single expandable card with both input and output.

Other SDK message types (system/init, stream events, auth status) are filtered out.

### Redux State

The `tasks` slice (`tasks-slice.ts`) manages:

- `tasks: TaskMetadata[]` — All task metadata, loaded from IPC on app init
- `selectedTaskId: string | null` — Currently viewed task
- `messagesByTask: Record<string, TaskMessage[]>` — Chat messages per task
- `streamingByTask: Record<string, boolean>` — Per-task streaming indicator
- `queuedMessagesByTask: Record<string, QueuedMessage[]>` — Messages queued while agent is busy (transient, not persisted)
- `pendingPermissions: PermissionRequest[]` — Pending permission dialogs

IPC event listeners in `stores/index.ts` dispatch actions for `task-updated`, `task-deleted`, `task-message`, `task-status-changed`, `task-permission-request`, and `task-error` events from the main process. The `task-status-changed` listener also handles auto-sending the next queued message when the agent transitions to `waiting`.

## UI Components

### Sidebar Task List

The sidebar's Tasks section (`tasks/task-list.tsx`) replaces the former placeholder:

- Header with "Tasks" label, archive toggle, and `+` button for creating new tasks
- Clicking `+` enters a "pending new task" state — the primary panel shows a site picker dropdown ("A new task for... [choose a site]") instead of an inline sidebar picker
- Task items show title, site name, and a status dot (blue pulsing = in-progress, gray = waiting, green = done)
- Archive button appears when hovering the status dot (not the whole row), replacing the dot
- Non-archived tasks sorted by `updatedAt` descending
- Archive toggle opens a Popover flyout listing archived tasks with count and "Clear all" button

### Chat Panel

When a task is selected, the primary panel renders `TaskChatPanel` instead of `SiteContentTabs`:

- **Message list** — User messages (right-aligned, themed) and assistant messages (left-aligned). Tool call messages are filtered out of the main conversation view — they only appear in the activity indicator.
- **Auto-scroll** — Scrolls to bottom on new messages
- **Streaming indicator** — Bouncing dots while agent is responding
- **Activity indicator** — Compact status bar above the chat input showing the agent's current state ("Thinking", "Editing", "Searching files", etc.) with an elapsed time counter. Clicking opens a flyout panel listing all activity (user messages, assistant responses, tool calls) with relative timestamps. Tool entries in the log are expandable to show full input/output details. Uses a blue pulsing dot only while the agent is actively streaming; no dot when idle.
- **Permission prompt** — Inline amber dialog above the input when the agent needs filesystem approval
- **Input** — Textarea with Enter-to-send (Shift+Enter for newlines). Supports image attachments via file picker, clipboard paste, or browser area capture. Images exceeding the API's 5MB base64 limit are automatically resized using a canvas (`resize-image.ts`).
- **Message queue** — Users can type and send follow-up messages while the agent is streaming. Queued messages appear as compact chips above the input. They auto-send one-by-one as the agent completes each turn. Clicking a chip restores it to the input for editing; the X button dismisses it. Auto-send pauses if the agent's last message was an error.

### Site Overview Integration

The site overview tab includes a "New task" button in the shortcuts section that creates a task pre-bound to that site.

## IPC Interface

### Handlers (invoke-style, return values)

| Handler | Description |
|---------|-------------|
| `createTask(siteId)` | Create task metadata, returns `TaskMetadata` |
| `getAllTasks()` | Returns all tasks |
| `updateTask(taskId, updates)` | Partial update (title, status, archived, sessionId) |
| `archiveTask(taskId)` | Set archived=true |
| `deleteTask(taskId)` | Remove from appdata |
| `clearArchivedTasks()` | Delete all archived tasks, returns removed IDs |
| `updateTaskStatus(taskId, status)` | Update status field |
| `startTaskAgentHandler(taskId, prompt, resumeSessionId?)` | Start agent session |
| `sendTaskMessageHandler(taskId, message)` | Send follow-up message |
| `interruptTaskHandler(taskId)` | Interrupt active agent |
| `respondToPermissionRequestHandler(requestId, response, taskId?)` | Resolve permission |

### Events (main -> renderer)

| Event | Payload | Description |
|-------|---------|-------------|
| `task-updated` | `TaskMetadata` | Task metadata changed |
| `task-deleted` | `string` (taskId) | Task removed |
| `task-message` | `{ taskId, message: TaskMessage }` | New chat message |
| `task-status-changed` | `{ taskId, status }` | Agent status transition |
| `task-permission-request` | `PermissionRequest` | Agent needs user approval |
| `task-error` | `{ taskId, error }` | Agent error |
| `browser-navigate` | `{ siteId, url }` | Agent navigated the browser (syncs preview iframe) |

## Key Files

```
apps/studio/src/
├── modules/ai/
│ ├── types.ts # TaskMetadata, TaskMessage, PermissionRequest
│ └── lib/
│ ├── ipc-handlers.ts # Task CRUD + agent lifecycle handlers
│ ├── agent-manager.ts # Active Query management, message loop, permissions
│ ├── tools.ts # Desktop MCP tools (site_list, wp_cli, etc.)
│ ├── browser-inspector.ts # Hidden BrowserWindow manager for agent inspection
│ ├── browser-tools.ts # Browser MCP tools (navigate, screenshot, etc.)
│ ├── provider-resolver.ts # Auth provider fallback (wpcom/claude/api-key)
│ └── message-serializer.ts # SDKMessage -> TaskMessage conversion
├── components/site-menu.tsx # Updated: clears task selection on site click
├── stores/
│ └── tasks-slice.ts # Redux state for tasks, messages, permissions
├── components/new-ui/tasks/
│ ├── task-list.tsx # Sidebar task list with + button and archive flyout
│ ├── task-list-item.tsx # Individual task item with status dot
│ ├── task-new-panel.tsx # New task site picker (primary panel)
│ ├── task-chat-panel.tsx # Chat panel (primary panel replacement)
│ ├── task-chat-input.tsx # Message input with agent IPC + queue-on-send
│ ├── task-queued-messages.tsx # Queued message chips (click to restore, X to dismiss)
│ ├── task-message-list.tsx # Message bubbles (user/assistant only, no tool cards)
│ ├── task-activity-indicator.tsx # Status bar + expandable activity log flyout
│ └── task-permission-prompt.tsx # Inline permission dialog
├── lib/resize-image.ts # Auto-resize base64 images exceeding API limit
├── storage/storage-types.ts # UserData.tasks field added
├── ipc-handlers.ts # Re-exports task handlers
├── ipc-utils.ts # Task IPC event types
├── preload.ts # Task IPC bridge methods
└── stores/index.ts # Tasks reducer + IPC event listeners

tools/common/ai/
└── system-prompt.ts # Shared system prompt (used by CLI and desktop)

apps/cli/ai/
└── system-prompt.ts # Re-exports from tools/common/ai/
```

## What's Next

- Session resume on app restart (sessionId is persisted, `query({ resume })` is wired)
- More MCP tools (site_create, site_delete, preview_create/update/delete, validate_blocks)
- Markdown rendering in assistant messages
- Streaming partial text (SDKPartialAssistantMessage events)
- ~~Auto-title refinement (use agent to generate a better title after first exchange)~~ Done — uses Haiku to generate titles
- Error recovery UI (provider not available, rate limits, etc.). Agent errors now send `ai:done` after `ai:error` so the desktop cleans up properly and follow-ups can resume via session ID.
- Keyboard shortcuts (Escape to interrupt, etc.)
Loading
Loading