Smoke Test Skill

Creates a new Mastra project using

create-mastra@<tag>

and performs smoke testing of the Mastra Studio in Chrome.

Usage

/smoke-test --directory <path> --name <project-name> --tag <version> [--pm <package-manager>] [--llm <provider>]
/smoke-test -d <path> -n <project-name> -t <version> [-p <package-manager>] [-l <provider>]

Parameters

Parameter	Short	Description	Required	Default
`--directory`	`-d`	Parent directory where project will be created	Yes	-
`--name`	`-n`	Project name (will be created as subdirectory)	Yes	-
`--tag`	`-t`	Version tag for create-mastra (e.g., `latest` , `alpha` , `0.10.6` )	Yes	-
`--pm`	`-p`	Package manager: `npm` , `yarn` , `pnpm` , or `bun`	No	`npm`
`--llm`	`-l`	LLM provider: `openai` , `anthropic` , `groq` , `google` , `cerebras` , `mistral`	No	`openai`

Examples

# Minimal (required params only)
/smoke-test -d ~/projects -n my-test-app -t latest

# Full specification
/smoke-test --directory ~/projects --name my-test-app --tag alpha --pm pnpm --llm anthropic

# Using short flags
/smoke-test -d ./projects -n smoke-test-app -t 0.10.6 -p bun -l openai

Step 0: Parameter Validation (MUST RUN FIRST)

CRITICAL: Before proceeding, parse the ARGUMENTS and validate:

Parse arguments from the ARGUMENTS string provided above
Check required parameters:
- ```
--directory
```
  or
```
-d
```
  : REQUIRED - fail if missing
- ```
--name
```
  or
```
-n
```
  : REQUIRED - fail if missing
- ```
--tag
```
  or
```
-t
```
  : REQUIRED - fail if missing
Apply defaults for optional parameters:
- ```
--pm
```
  or
```
-p
```
  : Default to
```
npm
```
  if not provided
- ```
--llm
```
  or
```
-l
```
  : Default to
```
openai
```
  if not provided
Validate values:
- ```
pm
```
  must be one of:
```
npm
```
  ,
```
yarn
```
  ,
```
pnpm
```
  ,
```
bun
```
- ```
llm
```
  must be one of:
```
openai
```
  ,
```
anthropic
```
  ,
```
groq
```
  ,
```
google
```
  ,
```
cerebras
```
  ,
```
mistral
```
- ```
directory
```
  must exist (or will be created)
- ```
name
```
  should be a valid directory name (no spaces, special chars)

If validation fails: Stop and show usage help with the missing/invalid parameters.

If
-h
or
--help
is passed: Show this usage information and stop.

Prerequisites

This skill requires the Claude-in-Chrome MCP server for browser automation. Ensure it's configured and running.

Execution Steps

Step 1: Create the Mastra Project

Run the create-mastra command with explicit parameters to avoid interactive prompts:

# For npm
npx create-mastra@<tag> <project-name> -c agents,tools,workflows,scorers -l <llmProvider> -e

# For yarn
yarn create mastra@<tag> <project-name> -c agents,tools,workflows,scorers -l <llmProvider> -e

# For pnpm
pnpm create mastra@<tag> <project-name> -c agents,tools,workflows,scorers -l <llmProvider> -e

# For bun
bunx create-mastra@<tag> <project-name> -c agents,tools,workflows,scorers -l <llmProvider> -e

Flags explained:

```
-c agents,tools,workflows,scorers
```
- Include all components
```
-l <provider>
```
- Set the LLM provider
```
-e
```
- Include example code

Being explicit with all parameters ensures the CLI runs non-interactively.

Wait for the installation to complete. This may take 1-2 minutes depending on network speed.

Step 2: Verify Project Structure

After creation, verify the project has:

```
package.json
```
with mastra dependencies
```
src/mastra/index.ts
```
exporting a Mastra instance
```
.env
```
file (may need to be created)

Step 2.5: Add Agent Network for Network Mode Testing

To enable Network mode testing, add an agent network configuration:

Create activity-agent.ts in
```
src/mastra/agents/
```
:

typescript

import { Agent } from '@mastra/core/agent';
import { Memory } from '@mastra/memory';

export const activityAgent = new Agent({
  id: 'activity-agent',
  name: 'Activity Agent',
  instructions: `You are a helpful activity planning assistant that suggests activities based on weather conditions.`,
  model: '<provider>/<model>', // e.g., 'openai/gpt-4o'
  memory: new Memory(),
});

Create planner-network.ts in
```
src/mastra/agents/
```
:

typescript

import { Agent } from '@mastra/core/agent';
import { Memory } from '@mastra/memory';
import { weatherAgent } from './weather-agent';
import { activityAgent } from './activity-agent';

export const plannerNetwork = new Agent({
  id: 'planner-network',
  name: 'Planner Network',
  instructions: `You are a coordinator that manages weather and activity agents.`,
  model: '<provider>/<model>',
  agents: { weatherAgent, activityAgent }, // This makes it a network agent
  memory: new Memory(), // Memory is REQUIRED for network agents
});

Update index.ts to register the new agents:

typescript

import { activityAgent } from './agents/activity-agent';
import { plannerNetwork } from './agents/planner-network';

// In Mastra config:
agents: { weatherAgent, activityAgent, plannerNetwork },

Note: Network mode requires

agents

property (sub-agents) and

memory

(mandatory).

Step 3: Configure Environment Variables

Based on the selected LLM provider, check for the required API key:

Provider	Required Environment Variable
openai	`OPENAI_API_KEY`
anthropic	`ANTHROPIC_API_KEY`
groq	`GROQ_API_KEY`
google	`GOOGLE_GENERATIVE_AI_API_KEY`
cerebras	`CEREBRAS_API_KEY`
mistral	`MISTRAL_API_KEY`

Check in this order:

Check global environment first: Run
```
echo $<ENV_VAR_NAME>
```
to see if the key is already set globally
- If set globally, the project will inherit it - no
```
.env
```
  file needed
- Skip to Step 4
Check project
.env
file: If not set globally, check if
```
.env
```
exists in the project and contains the key
Ask user only if needed: If the key is not available globally or in
```
.env
```
:
- Ask the user for the API key
- Create the
```
.env
```
  file with the provided key

Only check for the ONE key matching the selected provider - don't check for all providers.

Step 4: Start the Development Server

Navigate to the project directory and start the dev server:

cd <directory>/<project-name>
<packageManager> run dev

The server typically starts on

http://localhost:4111

. Wait for the server to be ready before proceeding.

Step 5: Smoke Test the Studio

Use the Chrome browser automation tools to test the Mastra Studio.

5.1 Initial Setup

Get browser context using
```
tabs_context_mcp
```
Create a new tab using
```
tabs_create_mcp
```
Navigate to
```
http://localhost:4111
```

5.2 Test Checklist

Perform the following smoke tests using the Chrome automation tools:

Navigation & Basic Loading

Studio loads successfully (page contains "Mastra Studio" or shows agents list)
Take a screenshot of the home page

Agents Page (

/agents

)

Navigate to agents page
Verify at least one agent is listed (the example agent from
```
--default
```
)
Take a screenshot

Agent Detail (

/agents/<agentId>/chat

)

Click on an agent to view details
Verify the agent overview panel loads
Verify model settings panel is visible
Take a screenshot

Agent Chat

Send a test message to the agent (e.g., "Hello, can you help me?")
Wait for response
Verify response appears in the chat
Take a screenshot of the conversation

Network Mode (

/agents/planner-network/chat

)

Navigate to the planner-network agent
Select "Network" in Chat Method settings
Send a message: "What activities can I do in [city] based on the weather?"
Verify network coordination (shows weatherAgent indicator)
Verify completion check shows success
Take a screenshot

Tools Page (

/tools

)

Navigate to tools page
Verify tools list loads (should show weatherTool)
Take a screenshot

Tool Execution (

/tools/weatherTool

)

Click on the weatherTool to open detail page
Find the city input field and enter a test city (e.g., "Tokyo")
Click Submit button
Wait for execution to complete
Verify JSON output appears with weather data (temp, condition, etc.)
Take a screenshot

Workflows Page (

/workflows

)

Navigate to workflows page
Verify workflows list loads (should show weatherWorkflow)
Take a screenshot

Workflow Execution (

/workflows/weatherWorkflow

)

Click on the weatherWorkflow to open detail page
Verify visual graph displays (shows workflow steps)
Find the city input field and enter a test city (e.g., "London")
Click Run button
Wait for execution to complete
Verify steps show success (green checkmarks)
Click to view JSON output modal
Verify execution details with timing appear
Take a screenshot

Settings Page (

/settings

)

Navigate to settings page
Verify settings page loads
Take a screenshot

Observability Page (

/observability

)

Navigate to observability page
Verify traces list shows recent activity (from previous tests)
Click on a trace to view details
Verify timeline view shows steps and timing
Take a screenshot

Scorers Page (

/scorers

)

Navigate to scorers page
Verify scorers list loads (shows 3 example scorers)
Take a screenshot

Scorer Detail (use direct URL navigation)

Navigate directly to
```
/scorers/completeness-scorer
```
(don't click - use URL navigation)
Verify scorer detail page loads with name, description, and scores table
Take a screenshot
Note: Use direct URL navigation for scorer details due to client-side routing timing issues

Additional Pages (verify load only)

Templates page (
```
/templates
```
) - Gallery of starter templates
Request Context page (
```
/request-context
```
) - JSON editor
Processors page (
```
/processors
```
) - Empty state OK
MCP Servers page (
```
/mcps
```
) - Empty state OK

5.3 Report Results

After completing all tests, provide a summary:

Total tests passed/failed
Any errors encountered
Screenshots captured
Recommendations for issues found

Quick Reference

Step	Action
Create Project	`cd <directory> && npx create-mastra@<tag> <name> -c agents,tools,workflows,scorers -l <provider> -e`
Install Deps	Automatic during creation
Set Env Vars	Check global env first, then `.env` , ask user only if needed
Start Server	`cd <directory>/<name> && npm run dev`
Studio URL	`http://localhost:4111`

Troubleshooting

Server won't start

Verify
```
.env
```
has required API key
Check if port 4111 is available
Try
```
<pm> install
```
to reinstall dependencies

Browser can't connect

Wait a few seconds for server to fully start
Check terminal for server ready message
Verify no firewall blocking localhost

Agent chat fails

Verify API key is valid
Check server logs for errors
Ensure LLM provider API is accessible

Studio Routes

Feature	Route
Agents	`/agents`
Workflows	`/workflows`
Tools	`/tools`
Scorers	`/scorers`
Observability	`/observability`
MCP Servers	`/mcps`
Processors	`/processors`
Templates	`/templates`
Request Context	`/request-context`
Settings	`/settings`

Notes

The
```
-e
```
flag includes example agents, making smoke testing meaningful
If the user doesn't specify an LLM provider, default to OpenAI as it's most common
Take screenshots at each major step for documentation/debugging
Keep the dev server running in the background during testing
Always use explicit flags (
```
-c
```
,
```
-l
```
,
```
-e
```
) to ensure non-interactive execution
Network mode requires the
```
agents
```
property AND
```
memory
```
in the Agent constructor
Scorer detail pages may have issues with browser automation but work manually
Observability traces appear automatically after running agents or workflows

smoke-test

NPX Install

Tags

SKILL.md Content

Smoke Test Skill

Usage

Parameters

Examples

Step 0: Parameter Validation (MUST RUN FIRST)

Prerequisites

Execution Steps

Step 1: Create the Mastra Project

Step 2: Verify Project Structure

Step 2.5: Add Agent Network for Network Mode Testing

Step 3: Configure Environment Variables

Step 4: Start the Development Server

Step 5: Smoke Test the Studio

5.1 Initial Setup

5.2 Test Checklist

5.3 Report Results

Quick Reference

Troubleshooting

Studio Routes

Notes