You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This adds plugin-aware MCP workflow telemetry for Studio's AI integrations. This is needed so we can track usage of the Studio MCP by external agents, eg. https://github.com/Automattic/build-with-wordpress
Changes include:
adding a hidden studio mcp --telemetry-group <group> option so different AI plugin integrations can identify themselves when they launch the Studio MCP server
extending the Studio CLI bump-stat types to allow plugin telemetry groups and workflow-stage stats
adding a new record_workflow_event MCP tool that records workflow milestones like theme-build-started, theme-build-completed, and theme-build-failed
threading the configured telemetry group through the MCP startup path so the tool can record events without the caller having to pass the plugin identity every time
updating the Studio AI UI/system prompt metadata so the new tool is surfaced cleanly in the built-in assistant
adding targeted MCP tool tests for configured and unconfigured telemetry behavior
This is designed to support plugin-grouped telemetry such as:
codex-plugin + theme-build-started
claude-code-plugin + theme-build-completed
Testing
Automated
npm test -- apps/cli/ai/tests/tools.test.ts
Manual
Launch Studio MCP with a telemetry group, for example:
studio mcp --telemetry-group codex-plugin
Connect an MCP client and confirm the record_workflow_event tool is available.
Call the tool with:
workflow: theme-build
stage: started
Confirm the tool returns a success message showing the plugin group and stat name.
Repeat with completed and failed stages and with another workflow such as site-build.
Launch studio mcp without --telemetry-group and confirm the tool reports that workflow telemetry was skipped because no plugin group was configured.
Notes
I also ran npx tsc -p apps/cli/tsconfig.json --noEmit, but it still reports an existing unrelated memfs test typing issue in apps/cli/lib/tests/snapshots.test.ts.
Hi @glendaviesnz , does this PR relate to any Linear issue? is necessary for Telex? Thank you!
When ready feel free to assign me or YOLO team as reviewer. If it's still in progress it would be great to move it as a draft.
Hi @glendaviesnz , does this PR relate to any Linear issue? is necessary for Telex? Thank you! When ready feel free to assign me or YOLO team as reviewer. If it's still in progress it would be great to move it as a draft.
Sorry @sejas, meant to add more detail, done now with a link to Linear issue. I am very open to alternative approaches to this, just let me know if you see a better way to handle it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes: https://linear.app/a8c/issue/STU-1530/add-telemetry-for-external-agents
Summary
This adds plugin-aware MCP workflow telemetry for Studio's AI integrations. This is needed so we can track usage of the Studio MCP by external agents, eg. https://github.com/Automattic/build-with-wordpress
Changes include:
studio mcp --telemetry-group <group>option so different AI plugin integrations can identify themselves when they launch the Studio MCP serverrecord_workflow_eventMCP tool that records workflow milestones liketheme-build-started,theme-build-completed, andtheme-build-failedThis is designed to support plugin-grouped telemetry such as:
codex-plugin+theme-build-startedclaude-code-plugin+theme-build-completedTesting
Automated
npm test -- apps/cli/ai/tests/tools.test.tsManual
studio mcp --telemetry-group codex-pluginrecord_workflow_eventtool is available.workflow: theme-buildstage: startedcompletedandfailedstages and with another workflow such assite-build.studio mcpwithout--telemetry-groupand confirm the tool reports that workflow telemetry was skipped because no plugin group was configured.Notes
npx tsc -p apps/cli/tsconfig.json --noEmit, but it still reports an existing unrelatedmemfstest typing issue inapps/cli/lib/tests/snapshots.test.ts.