Skip to content

Refine agent dispatcher and PBI dispatcher skills to prioritize PR creation over issues #411

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
p3dr0rv wants to merge 1 commit into
base: master
Choose a base branch
from
+75 −37
Open

Refine agent dispatcher and PBI dispatcher skills to prioritize PR creation over issues #411

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 16 additions & 19 deletions .github/agents/agent-dispatcher.agent.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,40 +14,37 @@ Read the skill file at `.github/skills/pbi-dispatcher/SKILL.md` and follow its w

## Key Rules

1. **Discover gh accounts first** — follow the GitHub Account Discovery sequence in the skill:
1. **Always create PRs, never bare issues** — The goal is to get Copilot coding agent to
generate a pull request with code changes. Use `create_pull_request_with_copilot` MCP
Copy link
Copy Markdown
Contributor

@shahzaibj shahzaibj Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the way it is worded it seems it is going to invoke each mechanism one by one. I believe that's not intended?

tool first, then `gh agent-task create` CLI, then generate a script. NEVER use
`gh issue create` as a dispatch mechanism.

2. **Discover gh accounts first** — follow the GitHub Account Discovery sequence in the skill:
- Check `.github/developer-local.json`
- Fall back to `gh auth status`
- Fall back to prompting the developer
- **Never hardcode GitHub usernames**

2. **Switch gh account** before any GitHub operations using the discovered usernames:
3. **Switch gh account** before any GitHub operations using the discovered usernames:
- `AzureAD/*` repos → `gh auth switch --user <discovered_public_username>`
- `identity-authnz-teams/*` repos → `gh auth switch --user <discovered_emu_username>`

2. **Read the full PBI** from ADO using `mcp_ado_wit_get_work_item` before dispatching

3. **Dispatch using `gh agent-task create`** with the FULL PBI description:
```powershell
gh auth switch --user <discovered_public_username>
$prompt = "<full PBI description including Fixes AB#ID>"
gh agent-task create $prompt --repo "OWNER/REPO" --base dev
```
4. **Read the full PBI** from ADO using `mcp_ado_wit_get_work_item` before dispatching

4. **Fallback** if `agent-task create` fails:
```powershell
gh issue create --repo "OWNER/REPO" --title "..." --body "..."
# Then assign:
echo '{"assignees":["copilot-swe-agent[bot]"]}' | gh api /repos/OWNER/REPO/issues/NUMBER/assignees --method POST --input -
```
5. **Dispatch priority order:**
Copy link
Copy Markdown
Contributor

@shahzaibj shahzaibj Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this isn't a dispatch priority order but rather a priority order for trying different mechanisms for dispatch

- **First**: Try `create_pull_request_with_copilot` MCP tool
Copy link
Copy Markdown
Contributor

@shahzaibj shahzaibj Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since you want to use MCP, then let's add the MCP tool to mcp.json under vscode folder

- **Second**: If MCP returns 401/403, try `gh agent-task create` CLI
- **Third**: If CLI unavailable (no pwsh), generate a `.ps1` script using `gh agent-task create`
- **NEVER**: Do NOT use `gh issue create` — issues don't trigger Copilot coding agent

5. **Respect dependencies** — don't dispatch if dependent PBIs haven't been implemented yet
6. **Respect dependencies** — don't dispatch if dependent PBIs haven't been implemented yet

6. **Report dispatch results** back in detail. For each dispatched PBI, include:
7. **Report dispatch results** back in detail. For each dispatched PBI, include:
- The AB# ID
- The target repo
- The PR number and URL (if available from the `gh agent-task create` output)
- The session URL (if available)

The orchestrator will use this information to update dashboard state and artifacts.

7. Return the dispatch summary with AB# IDs, repos, PR numbers (if available), and status
8. Return the dispatch summary with AB# IDs, repos, PR numbers (if available), and status
77 changes: 59 additions & 18 deletions .github/skills/pbi-dispatcher/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,30 @@ description: Dispatch Azure DevOps PBIs to GitHub Copilot coding agent for auton

# PBI Dispatcher

Dispatch Azure DevOps PBIs to GitHub Copilot coding agent by creating GitHub Issues in the
target repos and assigning them to `copilot-swe-agent[bot]`.
Dispatch Azure DevOps PBIs to GitHub Copilot coding agent by creating PRs via the
`create_pull_request_with_copilot` GitHub MCP tool (preferred) or `gh agent-task create` CLI.

**IMPORTANT**: Always create PRs, never issues. The goal is to get Copilot coding agent to
generate a pull request with code changes — NOT to create a GitHub Issue.

## Dispatch Method Priority

Try these methods **in order**. Use the first one that succeeds:

1. **`create_pull_request_with_copilot` MCP tool** (PREFERRED — works inside Copilot CLI)
2. **`assign_copilot_to_issue` MCP tool** (if an issue already exists)
3. **`gh agent-task create` CLI** (if MCP tools return 401/403 due to auth limitations)
4. **Generate a script** for the developer to run manually (last resort)

**NEVER create a bare GitHub Issue as the dispatch mechanism.** If the MCP tool fails with
401 (EMU auth can't access public repos), fall back to generating a `gh agent-task create`
script — NOT a `gh issue create` script.

## Prerequisites

- **ADO MCP Server** running (for reading PBI details)
- **GitHub CLI** (`gh`) authenticated with accounts for target repos
- **GitHub MCP Server** configured (for `create_pull_request_with_copilot`)
- **GitHub CLI** (`gh`) authenticated as fallback
- PBIs in ADO with tag `copilot-agent-ready`
- Copilot coding agent enabled on target repos

Expand Down Expand Up @@ -107,11 +124,31 @@ PBI description — it will be needed for the dispatch prompt.
### 2. Check Dependencies
For each PBI, check if its dependencies (other AB# IDs) have merged PRs. Skip blocked PBIs.

### 3. Switch gh Account + Dispatch to Copilot Agent
### 3. Dispatch to Copilot Agent

For each ready PBI, try these methods **in order**:

#### Method 1: `create_pull_request_with_copilot` MCP Tool (PREFERRED)

This is the most direct method — it creates a PR with Copilot coding agent in one step,
no shell commands needed:

```
create_pull_request_with_copilot(
Copy link
Copy Markdown
Contributor

@shahzaibj shahzaibj Apr 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this tool automatically assigns it to copilot? or does something need to be done to assign it to copilot?

Do you get copilot session id back that the orchestrator can render in the chat?

owner: "AzureAD", // or "identity-authnz-teams" for broker
repo: "microsoft-authentication-library-common-for-android",
base_ref: "dev",
title: "[PBI Title]",
problem_statement: "<full PBI description including 'Fixes AB#ID'>"
)
```

For each ready PBI:
**If this returns 401/403** (common when GitHub MCP uses EMU auth that can't access public
repos), fall back to Method 2.

**Step 1: Switch to the correct gh account** (using the discovered username from above):
#### Method 2: `gh agent-task create` CLI

**Step 1: Switch to the correct gh account** (using the discovered username):
```bash
# For AzureAD/* repos (common, msal, adal):
gh auth switch --user <discovered_public_username>
Expand All @@ -120,7 +157,7 @@ gh auth switch --user <discovered_public_username>
gh auth switch --user <discovered_emu_username>
```

**Step 2: Dispatch using `gh agent-task create` (PREFERRED — requires gh v2.80+):**
**Step 2: Dispatch:**

Write the full PBI description to a temp file and pipe it as the prompt. This avoids
shell escaping issues and ensures the full context reaches the agent:
Expand Down Expand Up @@ -152,20 +189,24 @@ gh agent-task create (Get-Content "$env:TEMP\pbi-prompt.txt" -Raw) --repo "OWNER
- Do NOT include local file paths (design-docs/, etc.) — the agent can't access them
- Do NOT truncate — the full description IS the implementation spec

**Step 3 (FALLBACK — if `gh agent-task create` fails):**
**If `gh agent-task create` also fails** (not installed, pwsh unavailable), fall back to
Method 3.

Create a GitHub Issue and assign to Copilot:
```bash
gh issue create \
--repo "OWNER/REPO" \
--title "[PBI Title]" \
--body "[Full PBI description with 'Fixes AB#ID']"
```
Then assign via API (extract issue number from the URL output):
```bash
echo '{"assignees":["copilot-swe-agent[bot]"],"agent_assignment":{"target_repo":"OWNER/REPO","base_branch":"dev","custom_instructions":"Follow copilot-instructions.md. PR title must include Fixes AB#ID."}}' | gh api /repos/OWNER/REPO/issues/ISSUE_NUMBER/assignees --method POST --input -
#### Method 3: Generate Script for Developer (LAST RESORT)

If neither MCP tools nor `gh` CLI are available from this environment, generate a
PowerShell script file the developer can run in their own terminal. The script MUST use
`gh agent-task create` (NOT `gh issue create`):

```powershell
# Save to a .ps1 file for the developer:
gh auth switch --user <discovered_public_username>
gh agent-task create "<prompt>" --repo "OWNER/REPO" --base dev
```

**NEVER generate a script that uses `gh issue create`.** Issues are not PRs — they don't
trigger Copilot coding agent to write code.

### 4. Update ADO State
Mark the ADO work item as `Active`, add tag `agent-dispatched`.

Expand Down
Loading