Skip to content

Commit 9e5afae

Browse files
robgruengithub-advanced-security[bot]Copilot
authored
Added onboarding agent (#2190)
This pull request introduces the new "onboarding-agent" package to the TypeAgent monorepo, providing a comprehensive agent for automating the integration of new applications or APIs into TypeAgent. It includes documentation, package setup, and the initial implementation of the Discovery phase (Phase 1) with its schema and grammar. Additionally, there are improvements to the agent server session manager to pre-initialize session dispatchers for better performance. For BEST results: register TypeAgent as an MCP server with your AI of choice (Github Copilot, Claude, etc.) and then use that to leverage the onboarding agent: ``` create a new typeagent agent for cmd.exe using the onboarding agent ``` --------- Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: robgruen <25374553+robgruen@users.noreply.github.com>
1 parent c6e4927 commit 9e5afae

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+7049
-17
lines changed
Lines changed: 321 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,321 @@
1+
# Agent Patterns — Architecture & Design
2+
3+
> **Scope:** This document describes the nine architectural patterns used by
4+
> TypeAgent application agents. Use it when designing a new agent or choosing
5+
> a scaffolding template. For the dispatcher that hosts agents, see
6+
> `dispatcher.md`. For how to automate agent creation, see the onboarding
7+
> agent at `packages/agents/onboarding/`.
8+
9+
## Overview
10+
11+
Every TypeAgent agent exports an `instantiate(): AppAgent` function and
12+
implements the `AppAgent` interface from `@typeagent/agent-sdk`. Beyond that
13+
common contract, agents fall into nine patterns based on how they communicate
14+
with external systems, whether they manage persistent state, and what kind of
15+
output they produce.
16+
17+
| Pattern | When to use | Examples |
18+
| ------------------------ | ---------------------------------------- | ------------------------------- |
19+
| `schema-grammar` | Bounded set of typed actions (default) | `weather`, `photo`, `list` |
20+
| `external-api` | Authenticated REST / cloud API | `calendar`, `email`, `player` |
21+
| `llm-streaming` | Agent calls an LLM, streams results | `chat`, `greeting` |
22+
| `sub-agent-orchestrator` | API surface too large for one schema | `desktop`, `code`, `browser` |
23+
| `websocket-bridge` | Automate a host app via a plugin | `browser`, `code` |
24+
| `state-machine` | Multi-phase workflow with approval gates | `onboarding`, `scriptflow` |
25+
| `native-platform` | OS / device APIs, no cloud | `androidMobile`, `playerLocal` |
26+
| `view-ui` | Rich interactive web-view UI | `turtle`, `montage`, `markdown` |
27+
| `command-handler` | Simple settings-style direct dispatch | `settings`, `test` |
28+
29+
---
30+
31+
## Pattern details
32+
33+
### 1. `schema-grammar` — Standard (default)
34+
35+
The canonical TypeAgent pattern. Define TypeScript action types, generate a
36+
`.agr` grammar file for natural language matching, and implement a typed
37+
dispatch handler.
38+
39+
**File layout**
40+
41+
```
42+
src/
43+
<name>Manifest.json ← agent metadata, schema pointers
44+
<name>Schema.ts ← exported union of action types
45+
<name>Schema.agr ← grammar rules (NL → action)
46+
<name>ActionHandler.ts ← instantiate(); executeAction() dispatch
47+
```
48+
49+
**AppAgent lifecycle**
50+
51+
```typescript
52+
export function instantiate(): AppAgent {
53+
return { initializeAgentContext, executeAction };
54+
}
55+
```
56+
57+
**When to choose:** any integration with a well-defined, enumerable set of
58+
actions — REST APIs, CLI tools, file operations, data queries.
59+
60+
**Examples:** `weather`, `photo`, `list`, `image`, `video`
61+
62+
---
63+
64+
### 2. `external-api` — REST / OAuth Bridge
65+
66+
Extends the standard pattern with an API client class and token persistence.
67+
The handler creates a client on `initializeAgentContext` and authenticates
68+
lazily or eagerly on `updateAgentContext`.
69+
70+
**Additional files**
71+
72+
```
73+
src/
74+
<name>Bridge.ts ← API client class with auth + HTTP methods
75+
~/.typeagent/profiles/<profile>/<name>/token.json ← persisted OAuth token
76+
```
77+
78+
**Manifest flags:** none beyond standard.
79+
80+
**When to choose:** cloud services requiring OAuth or API-key auth — MS Graph,
81+
Spotify, GitHub, Slack, etc.
82+
83+
**Examples:** `calendar` (MS Graph), `email` (MS Graph + Google), `player`
84+
(Spotify)
85+
86+
---
87+
88+
### 3. `llm-streaming` — LLM-Injected / Streaming
89+
90+
Runs inside the dispatcher process rather than as a sandboxed plugin
91+
(`injected: true`). The handler calls an LLM directly and streams partial
92+
results back to the client via `streamingActionContext`.
93+
94+
**Manifest flags**
95+
96+
```json
97+
{
98+
"injected": true,
99+
"cached": false,
100+
"streamingActions": ["generateResponse"]
101+
}
102+
```
103+
104+
**Dependencies added:** `aiclient`, `typechat`
105+
106+
**When to choose:** conversational or generative agents that need to produce
107+
streaming text — chat assistants, summarizers, code generators.
108+
109+
**Examples:** `chat`, `greeting`
110+
111+
---
112+
113+
### 4. `sub-agent-orchestrator` — Multiple Sub-Schemas
114+
115+
A root agent with a `subActionManifests` map in its manifest. Each sub-schema
116+
has its own TypeScript types, grammar file, and handler module. The root
117+
`executeAction` routes to the appropriate module based on action name (each
118+
sub-schema owns a disjoint set of names).
119+
120+
**File layout**
121+
122+
```
123+
src/
124+
<name>Manifest.json ← includes subActionManifests map
125+
<name>Schema.ts ← root union type (optional)
126+
<name>ActionHandler.ts ← routes to sub-handlers
127+
actions/
128+
<group>ActionsSchema.ts ← per-group action types
129+
<group>ActionsSchema.agr ← per-group grammar
130+
```
131+
132+
**Manifest structure**
133+
134+
```json
135+
{
136+
"subActionManifests": {
137+
"groupOne": { "schema": { ... } },
138+
"groupTwo": { "schema": { ... } }
139+
}
140+
}
141+
```
142+
143+
**When to choose:** integrations whose API surface spans distinct domains that
144+
would make a single schema unwieldy — editor + debugger + terminal, or
145+
read/write/admin operations.
146+
147+
**Examples:** `desktop` (7 sub-agents), `code` (6), `browser` (4), `onboarding` (7)
148+
149+
---
150+
151+
### 5. `websocket-bridge` — Host Plugin via WebSocket
152+
153+
The TypeAgent handler owns a `WebSocketServer`. A host-side plugin (VS Code
154+
extension, browser extension, Electron renderer, Office add-in) connects as
155+
the WebSocket client. Commands flow TypeAgent → WebSocket → plugin; results
156+
flow back. Requires a companion plugin project.
157+
158+
**File layout**
159+
160+
```
161+
src/
162+
<name>ActionHandler.ts ← starts WebSocketServer, forwards actions
163+
<name>Bridge.ts ← WebSocket server + pending-request map
164+
plugin/ (or extension/)
165+
<host-specific files> ← connects to the bridge and calls host APIs
166+
```
167+
168+
**AppAgent lifecycle:** implements `closeAgentContext()` to stop the server.
169+
170+
**Dependencies added:** `ws`
171+
172+
**When to choose:** automating an application that runs its own JS/TS runtime
173+
(VS Code, Electron, browser, Office).
174+
175+
**Examples:** `browser`, `code`
176+
177+
---
178+
179+
### 6. `state-machine` — Multi-Phase Workflow
180+
181+
Persists phase state to `~/.typeagent/<name>/<workflowId>/state.json`. Each
182+
phase progresses `pending → in-progress → approved` and must be approved
183+
before the next phase begins. Designed for long-running automation that spans
184+
multiple sessions.
185+
186+
**State structure**
187+
188+
```typescript
189+
type WorkflowState = {
190+
workflowId: string;
191+
currentPhase: string;
192+
phases: Record<string, { status: PhaseStatus; updatedAt: string }>;
193+
config: Record<string, unknown>;
194+
createdAt: string;
195+
updatedAt: string;
196+
};
197+
```
198+
199+
**When to choose:** build pipelines, onboarding flows, multi-step test
200+
suites — any workflow where a human must review and approve each stage before
201+
proceeding.
202+
203+
**Examples:** `onboarding`, `scriptflow`, `taskflow`
204+
205+
---
206+
207+
### 7. `native-platform` — OS / Device Automation
208+
209+
Invokes platform APIs directly via `child_process.exec` / `spawn`, device
210+
SDKs, or signal handling. No cloud dependency.
211+
212+
**Key considerations**
213+
214+
- Branch on `process.platform` (`"win32"` / `"darwin"` / `"linux"`) for
215+
cross-platform commands.
216+
- Use `SIGSTOP` / `SIGCONT` for pause/resume on Unix where applicable.
217+
- Keep side effects narrow — prefer reversible commands.
218+
219+
**When to choose:** controlling a desktop application, mobile device, or
220+
system service that exposes no REST API.
221+
222+
**Examples:** `androidMobile`, `playerLocal` (macOS `afplay` / Linux `mpv`),
223+
`desktop`
224+
225+
---
226+
227+
### 8. `view-ui` — Web View Renderer
228+
229+
A minimal action handler that opens a local HTTP server serving a `site/`
230+
directory and signals the dispatcher to open the view via `openLocalView`.
231+
The actual UX lives in the `site/` directory; the handler communicates with
232+
it via display APIs and IPC types.
233+
234+
**File layout**
235+
236+
```
237+
src/
238+
<name>ActionHandler.ts ← opens/closes view, handles actions
239+
ipcTypes.ts ← shared message types for handler ↔ view IPC
240+
site/
241+
index.html ← web view entry point
242+
...
243+
```
244+
245+
**Manifest flags:** `"localView": true`
246+
247+
**When to choose:** agents that need a rich interactive UI beyond simple text
248+
or markdown output.
249+
250+
**Examples:** `turtle`, `montage`, `markdown`
251+
252+
---
253+
254+
### 9. `command-handler` — Direct Dispatch
255+
256+
Uses a `handlers` map keyed by action name instead of the typed `executeAction`
257+
pipeline. Actions map directly to named handler functions. The pattern is
258+
suited for agents with a small number of well-known, settings-style commands
259+
where the full schema + grammar machinery adds more overhead than value.
260+
261+
```typescript
262+
export function instantiate(): AppAgent {
263+
return getCommandInterface(handlers);
264+
}
265+
266+
const handlers = {
267+
setSetting: async (params) => {
268+
/* ... */
269+
},
270+
getSetting: async (params) => {
271+
/* ... */
272+
},
273+
};
274+
```
275+
276+
**When to choose:** configuration agents, toggle-style controls, admin tools.
277+
278+
**Examples:** `settings`, `test`
279+
280+
---
281+
282+
## Choosing a pattern
283+
284+
```
285+
Does the agent need to stream text from an LLM?
286+
└─ Yes → llm-streaming
287+
288+
Does the agent automate an app with its own JS/TS runtime?
289+
└─ Yes → websocket-bridge
290+
291+
Does the agent span a multi-step, human-gated workflow?
292+
└─ Yes → state-machine
293+
294+
Is the API surface too large for one schema?
295+
└─ Yes → sub-agent-orchestrator
296+
297+
Does the agent need a rich interactive UI?
298+
└─ Yes → view-ui
299+
300+
Does the agent call an authenticated cloud API?
301+
└─ Yes → external-api
302+
303+
Does the agent invoke OS/device APIs directly?
304+
└─ Yes → native-platform
305+
306+
Does the agent have only a handful of well-known commands?
307+
└─ Yes → command-handler
308+
309+
Otherwise → schema-grammar (default)
310+
```
311+
312+
## Scaffolding
313+
314+
The onboarding agent's scaffolder can generate boilerplate for any pattern:
315+
316+
```
317+
scaffold the <name> agent using the <pattern> pattern
318+
```
319+
320+
Or use `list agent patterns` at runtime for the full table. See
321+
`packages/agents/onboarding/` for details.

ts/package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@
6262
"prettier": "^3.5.3",
6363
"shx": "^0.4.0"
6464
},
65-
"packageManager": "pnpm@10.32.1+sha512.a706938f0e89ac1456b6563eab4edf1d1faf3368d1191fc5c59790e96dc918e4456ab2e67d613de1043d2e8c81f87303e6b40d4ffeca9df15ef1ad567348f2be",
65+
"packageManager": "pnpm@10.33.0+sha512.10568bb4a6afb58c9eb3630da90cc9516417abebd3fabbe6739f0ae795728da1491e9db5a544c76ad8eb7570f5c4bb3d6c637b2cb41bfdcdb47fa823c8649319",
6666
"engines": {
6767
"node": ">=20.18.1",
6868
"pnpm": ">=10"

ts/packages/agentServer/server/src/server.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@ async function main() {
6363
// Pre-initialize the default session dispatcher before accepting clients,
6464
// so the first joinSession call is fast and concurrent joinSession calls
6565
// don't race to initialize the same dispatcher.
66-
await sessionManager.prewarmDefaultSession();
66+
await sessionManager.prewarmMostRecentSession();
6767

6868
const portIdx = process.argv.indexOf("--port");
6969
const port =

ts/packages/agentServer/server/src/sessionManager.ts

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -52,11 +52,11 @@ export type SessionManager = {
5252
*/
5353
resolveSessionId(sessionId: string | undefined): Promise<string>;
5454
/**
55-
* Pre-initialize the default session's dispatcher so it is ready before
56-
* the first client connects. If no sessions exist, a "default" session is
57-
* created. Safe to call multiple times.
55+
* Pre-initialize the most recently active session's dispatcher so it is
56+
* ready before the first client connects. If no sessions exist, a "default"
57+
* session is created. Safe to call multiple times.
5858
*/
59-
prewarmDefaultSession(): Promise<void>;
59+
prewarmMostRecentSession(): Promise<void>;
6060
joinSession(
6161
sessionId: string,
6262
clientIO: ClientIO,
@@ -173,7 +173,6 @@ export async function createSessionManager(
173173
createSharedDispatcher(hostName, {
174174
...baseOptions,
175175
persistDir,
176-
instanceDir: baseDir, // global instance root — shared across all server sessions
177176
persistSession: true,
178177
}),
179178
)
@@ -303,7 +302,7 @@ export async function createSessionManager(
303302
return info.sessionId;
304303
},
305304

306-
async prewarmDefaultSession(): Promise<void> {
305+
async prewarmMostRecentSession(): Promise<void> {
307306
const sessionId = await manager.resolveSessionId(undefined);
308307
const record = sessions.get(sessionId)!;
309308
cancelIdleTimer(record);

0 commit comments

Comments
 (0)