Back to Details

playwright-e2e-tests

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Playwright E2E Tests

Playwright E2E测试

Project Layout

项目结构

  • Tests: 
    web/tests/e2e/
    — organized by feature (
    auth/
    , 
    admin/
    , 
    chat/
    , 
    assistants/
    , 
    connectors/
    , 
    mcp/
    )
  • Config: 
    web/playwright.config.ts
  • Utilities: 
    web/tests/e2e/utils/
  • Constants: 
    web/tests/e2e/constants.ts
  • Global setup: 
    web/tests/e2e/global-setup.ts
  • Output: 
    web/output/playwright/
  • 测试文件：
    web/tests/e2e/
    — 按功能模块组织（
    auth/
    、
    admin/
    、
    chat/
    、
    assistants/
    、
    connectors/
    、
    mcp/
    ）
  • 配置文件：
    web/playwright.config.ts
  • 工具函数：
    web/tests/e2e/utils/
  • 常量定义：
    web/tests/e2e/constants.ts
  • 全局初始化：
    web/tests/e2e/global-setup.ts
  • 输出目录：
    web/output/playwright/

Imports

导入规则

Always use absolute imports with the 
@tests/e2e/
prefix — never relative paths (
../
, 
../../
). The alias is defined in 
web/tsconfig.json
and resolves to 
web/tests/
.
typescript
import { loginAs } from "@tests/e2e/utils/auth";
import { OnyxApiClient } from "@tests/e2e/utils/onyxApiClient";
import { TEST_ADMIN_CREDENTIALS } from "@tests/e2e/constants";
All new files should be 
.ts
, not 
.js
.
始终使用带有
@tests/e2e/
前缀的绝对导入——切勿使用相对路径（
../
、
../../
）。该别名在
web/tsconfig.json
中定义，指向
web/tests/
目录。
typescript
import { loginAs } from "@tests/e2e/utils/auth";
import { OnyxApiClient } from "@tests/e2e/utils/onyxApiClient";
import { TEST_ADMIN_CREDENTIALS } from "@tests/e2e/constants";
所有新文件都应使用
.ts
格式，而非
.js
。

Running Tests

运行测试

bash
undefined
bash
undefined

Run a specific test file

运行指定测试文件

npx playwright test web/tests/e2e/chat/default_assistant.spec.ts
npx playwright test web/tests/e2e/chat/default_assistant.spec.ts

Run a specific project

运行指定测试项目

npx playwright test --project admin npx playwright test --project exclusive
undefined
npx playwright test --project admin npx playwright test --project exclusive
undefined

Test Projects

测试项目

ProjectDescriptionParallelism
admin
Standard tests (excludes 
@exclusive
)		Parallel
exclusive
Serial, slower tests (tagged 
@exclusive
)		1 worker
All tests use 
admin_auth.json
storage state by default (pre-authenticated admin session).
项目名称描述并行执行
admin
标准测试（排除标记
@exclusive
的测试）		支持并行
exclusive
串行执行的慢测试（标记
@exclusive
）		1个工作进程
所有测试默认使用
admin_auth.json
存储状态（预认证的管理员会话）。

Authentication

认证机制

Global setup (
global-setup.ts
) runs automatically before all tests and handles:
  • Server readiness check (polls health endpoint, 60s timeout)
  • Provisioning test users: admin, admin2, and a pool of worker users (
    worker0@example.com
    through 
    worker7@example.com
    ) (idempotent)
  • API login + saving storage states: 
    admin_auth.json
    , 
    admin2_auth.json
    , and 
    worker{N}_auth.json
    for each worker user
  • Setting display name to 
    "worker"
    for each worker user
  • Promoting admin2 to admin role
  • Ensuring a public LLM provider exists
Both test projects set 
storageState: "admin_auth.json"
, so every test starts pre-authenticated as admin with no login code needed.
When a test needs a different user, use API-based login — never drive the login UI:
typescript
import { loginAs } from "@tests/e2e/utils/auth";

await page.context().clearCookies();
await loginAs(page, "admin2");

// Log in as the worker-specific user (preferred for test isolation):
import { loginAsWorkerUser } from "@tests/e2e/utils/auth";
await page.context().clearCookies();
await loginAsWorkerUser(page, testInfo.workerIndex);
全局初始化脚本（
global-setup.ts
）会在所有测试运行前自动执行，负责：
  • 服务器就绪检查（轮询健康检查端点，超时时间60秒）
  • 预创建测试用户：管理员、admin2，以及一组工作用户池（
    worker0@example.com
    至
    worker7@example.com
    ）（操作具有幂等性）
  • API登录并保存存储状态：
    admin_auth.json
    、
    admin2_auth.json
    ，以及每个工作用户对应的
    worker{N}_auth.json
  • 为每个工作用户设置显示名称为
    "worker"
  • 将admin2提升为管理员角色
  • 确保存在一个公开的LLM提供商
两个测试项目均设置
storageState: "admin_auth.json"
，因此每个测试启动时都已以管理员身份预认证，无需编写登录代码。
当测试需要使用其他用户时，使用基于API的登录——切勿通过UI驱动登录：
typescript
import { loginAs } from "@tests/e2e/utils/auth";

await page.context().clearCookies();
await loginAs(page, "admin2");

// 登录为特定工作用户（推荐用于测试隔离）：
import { loginAsWorkerUser } from "@tests/e2e/utils/auth";
await page.context().clearCookies();
await loginAsWorkerUser(page, testInfo.workerIndex);

Test Structure

测试结构

Tests start pre-authenticated as admin — navigate and test directly:
typescript
import { test, expect } from "@playwright/test";

test.describe("Feature Name", () => {
  test("should describe expected behavior clearly", async ({ page }) => {
    await page.goto("/app");
    await page.waitForLoadState("networkidle");
    // Already authenticated as admin — go straight to testing
  });
});
User isolation — tests that modify visible app state (creating assistants, sending chat messages, pinning items) should run as a worker-specific user and clean up resources in 
afterAll
. Global setup provisions a pool of worker users (
worker0@example.com
through 
worker7@example.com
). 
loginAsWorkerUser
maps 
testInfo.workerIndex
to a pool slot via modulo, so retry workers (which get incrementing indices beyond the pool size) safely reuse existing users. This ensures parallel workers never share user state, keeps usernames deterministic for screenshots, and avoids cross-contamination:
typescript
import { test } from "@playwright/test";
import { loginAsWorkerUser } from "@tests/e2e/utils/auth";

test.beforeEach(async ({ page }, testInfo) => {
  await page.context().clearCookies();
  await loginAsWorkerUser(page, testInfo.workerIndex);
});
If the test requires admin privileges and modifies visible state, use 
"admin2"
instead — it's a pre-provisioned admin account that keeps the primary 
"admin"
clean for other parallel tests. Switch to 
"admin"
only for privileged setup (creating providers, configuring tools), then back to the worker user for the actual test. See 
chat/default_assistant.spec.ts
for a full example.
loginAsRandomUser
exists for the rare case where the test requires a brand-new user (e.g. onboarding flows). Avoid it elsewhere — it produces non-deterministic usernames that complicate screenshots.
API resource setup — only when tests need to create backend resources (image gen configs, web search providers, MCP servers). Use 
beforeAll
/
afterAll
with 
OnyxApiClient
to create and clean up. See 
chat/default_assistant.spec.ts
or 
mcp/mcp_oauth_flow.spec.ts
for examples. This is uncommon (~4 of 37 test files).
测试启动时已以管理员身份预认证——可直接导航并执行测试：
typescript
import { test, expect } from "@playwright/test";

test.describe("功能名称", () => {
  test("应清晰描述预期行为", async ({ page }) => {
    await page.goto("/app");
    await page.waitForLoadState("networkidle");
    // 已认证为管理员——直接开始测试
  });
});
用户隔离——对于会修改可见应用状态的测试（创建助手、发送聊天消息、固定项目等），应使用特定工作用户运行，并在
afterAll
中清理资源。全局初始化脚本预创建了一组工作用户池（
worker0@example.com
至
worker7@example.com
）。
loginAsWorkerUser
通过取模运算将
testInfo.workerIndex
映射到用户池中的某个用户，因此重试的工作进程（索引会超出用户池大小）可以安全地重用现有用户。这确保了并行工作进程永远不会共享用户状态，保持用户名的确定性以便截图，避免测试间的相互干扰：
typescript
import { test } from "@playwright/test";
import { loginAsWorkerUser } from "@tests/e2e/utils/auth";

test.beforeEach(async ({ page }, testInfo) => {
  await page.context().clearCookies();
  await loginAsWorkerUser(page, testInfo.workerIndex);
});
如果测试需要管理员权限且会修改可见状态，请改用
"admin2"
——这是一个预创建的管理员账户，可保持主
"admin"
账户干净，供其他并行测试使用。仅在需要特权设置（创建提供商、配置工具）时切换到
"admin"
，然后切换回工作用户执行实际测试。完整示例请参考
chat/default_assistant.spec.ts
。
loginAsRandomUser
仅适用于测试需要全新用户的罕见场景（例如引导流程）。其他场景请避免使用——它会生成非确定性的用户名，给截图带来麻烦。
API资源设置——仅当测试需要创建后端资源（图像生成配置、网页搜索提供商、MCP服务器）时使用。使用
beforeAll
/
afterAll
结合
OnyxApiClient
来创建和清理资源。示例请参考
chat/default_assistant.spec.ts
或
mcp/mcp_oauth_flow.spec.ts
。这种情况并不常见（37个测试文件中约有4个）。

Key Utilities

核心工具函数

OnyxApiClient
(
@tests/e2e/utils/onyxApiClient
)

OnyxApiClient
（
@tests/e2e/utils/onyxApiClient
）

Backend API client for test setup/teardown. Key methods:
  • Connectors: 
    createFileConnector()
    , 
    deleteCCPair()
    , 
    pauseConnector()
  • LLM Providers: 
    ensurePublicProvider()
    , 
    createRestrictedProvider()
    , 
    setProviderAsDefault()
  • Assistants: 
    createAssistant()
    , 
    deleteAssistant()
    , 
    findAssistantByName()
  • User Groups: 
    createUserGroup()
    , 
    deleteUserGroup()
    , 
    setUserRole()
  • Tools: 
    createWebSearchProvider()
    , 
    createImageGenerationConfig()
  • Chat: 
    createChatSession()
    , 
    deleteChatSession()
用于测试初始化/清理的后端API客户端。核心方法：
  • 连接器：
    createFileConnector()
    、
    deleteCCPair()
    、
    pauseConnector()
  • LLM提供商：
    ensurePublicProvider()
    、
    createRestrictedProvider()
    、
    setProviderAsDefault()
  • 助手：
    createAssistant()
    、
    deleteAssistant()
    、
    findAssistantByName()
  • 用户组：
    createUserGroup()
    、
    deleteUserGroup()
    、
    setUserRole()
  • 工具：
    createWebSearchProvider()
    、
    createImageGenerationConfig()
  • 聊天：
    createChatSession()
    、
    deleteChatSession()

chatActions
(
@tests/e2e/utils/chatActions
)

chatActions
（
@tests/e2e/utils/chatActions
）

  • sendMessage(page, message)
    — sends a message and waits for AI response
  • startNewChat(page)
    — clicks new-chat button and waits for intro
  • verifyDefaultAssistantIsChosen(page)
    — checks Onyx logo is visible
  • verifyAssistantIsChosen(page, name)
    — checks assistant name display
  • switchModel(page, modelName)
    — switches LLM model via popover
  • sendMessage(page, message)
    — 发送消息并等待AI回复
  • startNewChat(page)
    — 点击新建聊天按钮并等待引导界面加载
  • verifyDefaultAssistantIsChosen(page)
    — 检查Onyx logo是否可见
  • verifyAssistantIsChosen(page, name)
    — 检查助手名称是否显示
  • switchModel(page, modelName)
    — 通过弹出框切换LLM模型

visualRegression
(
@tests/e2e/utils/visualRegression
)

visualRegression
（
@tests/e2e/utils/visualRegression
）

  • expectScreenshot(page, { name, mask?, hide?, fullPage? })
  • expectElementScreenshot(locator, { name, mask?, hide? })
  • Controlled by 
    VISUAL_REGRESSION=true
    env var
  • expectScreenshot(page, { name, mask?, hide?, fullPage? })
  • expectElementScreenshot(locator, { name, mask?, hide? })
  • 由环境变量
    VISUAL_REGRESSION=true
    控制

theme
(
@tests/e2e/utils/theme
)

theme
（
@tests/e2e/utils/theme
）

  • THEMES
    — 
    ["light", "dark"] as const
    array for iterating over both themes
  • setThemeBeforeNavigation(page, theme)
    — sets 
    next-themes
    theme via 
    localStorage
    before navigation
When tests need light/dark screenshots, loop over 
THEMES
at the 
test.describe
level and call 
setThemeBeforeNavigation
in 
beforeEach
before any 
page.goto()
. Include the theme in screenshot names. See 
admin/admin_pages.spec.ts
or 
chat/chat_message_rendering.spec.ts
for examples:
typescript
import { THEMES, setThemeBeforeNavigation } from "@tests/e2e/utils/theme";

for (const theme of THEMES) {
  test.describe(`Feature (${theme} mode)`, () => {
    test.beforeEach(async ({ page }) => {
      await setThemeBeforeNavigation(page, theme);
    });

    test("renders correctly", async ({ page }) => {
      await page.goto("/app");
      await expectScreenshot(page, { name: `feature-${theme}` });
    });
  });
}
  • THEMES
    — 包含
    ["light", "dark"]
    的常量数组，用于遍历两种主题
  • setThemeBeforeNavigation(page, theme)
    — 在导航前通过
    localStorage
    设置
    next-themes
    主题
当测试需要生成亮色/暗色模式截图时，在
test.describe
级别遍历
THEMES
，并在
beforeEach
中导航前调用
setThemeBeforeNavigation
。截图名称中需包含主题信息。示例请参考
admin/admin_pages.spec.ts
或
chat/chat_message_rendering.spec.ts
：
typescript
import { THEMES, setThemeBeforeNavigation } from "@tests/e2e/utils/theme";

for (const theme of THEMES) {
  test.describe(`功能（${theme}模式）`, () => {
    test.beforeEach(async ({ page }) => {
      await setThemeBeforeNavigation(page, theme);
    });

    test("渲染正常", async ({ page }) => {
      await page.goto("/app");
      await expectScreenshot(page, { name: `feature-${theme}` });
    });
  });
}

tools
(
@tests/e2e/utils/tools
)

tools
（
@tests/e2e/utils/tools
）

  • TOOL_IDS
    — centralized 
    data-testid
    selectors for tool options
  • openActionManagement(page)
    — opens the tool management popover
  • TOOL_IDS
    — 工具选项的集中式
    data-testid
    选择器
  • openActionManagement(page)
    — 打开工具管理弹出框

Locator Strategy

定位器策略

Use locators in this priority order:
  1. data-testid
    / 
    aria-label
     — preferred for Onyx components
    typescript
    page.getByTestId("AppSidebar/new-session")
page.getByLabel("admin-page-title")
  2. Role-based — for standard HTML elements
    typescript
    page.getByRole("button", { name: "Create" })
page.getByRole("dialog")
  3. Text/Label — for visible text content
    typescript
    page.getByText("Custom Assistant")
page.getByLabel("Email")
  4. CSS selectors — last resort, only when above won't work
    typescript
    page.locator('input[name="name"]')
page.locator("#onyx-chat-input-textarea")
Never use 
page.locator
with complex CSS/XPath when a built-in locator works.
按以下优先级使用定位器：
  1. data-testid
    / 
    aria-label
     — 优先用于Onyx组件
    typescript
    page.getByTestId("AppSidebar/new-session")
page.getByLabel("admin-page-title")
  2. 基于角色 — 用于标准HTML元素
    typescript
    page.getByRole("button", { name: "Create" })
page.getByRole("dialog")
  3. 文本/标签 — 用于可见文本内容
    typescript
    page.getByText("Custom Assistant")
page.getByLabel("Email")
  4. CSS选择器 — 最后选择，仅当以上方法都不适用时使用
    typescript
    page.locator('input[name="name"]')
page.locator("#onyx-chat-input-textarea")
切勿在内置定位器可用的情况下，使用复杂CSS/XPath的
page.locator
。

Assertions

断言

Use web-first assertions — they auto-retry until the condition is met:
typescript
// Visibility
await expect(page.getByTestId("onyx-logo")).toBeVisible({ timeout: 5000 });

// Text content
await expect(page.getByTestId("assistant-name-display")).toHaveText("My Assistant");

// Count
await expect(page.locator('[data-testid="onyx-ai-message"]')).toHaveCount(2, { timeout: 30000 });

// URL
await expect(page).toHaveURL(/chatId=/);

// Element state
await expect(toggle).toBeChecked();
await expect(button).toBeEnabled();
Never use 
assert
statements or hardcoded 
page.waitForTimeout()
.
使用Web优先断言——它们会自动重试直到条件满足：
typescript
// 可见性
await expect(page.getByTestId("onyx-logo")).toBeVisible({ timeout: 5000 });

// 文本内容
await expect(page.getByTestId("assistant-name-display")).toHaveText("My Assistant");

// 数量
await expect(page.locator('[data-testid="onyx-ai-message"]')).toHaveCount(2, { timeout: 30000 });

// URL
await expect(page).toHaveURL(/chatId=/);

// 元素状态
await expect(toggle).toBeChecked();
await expect(button).toBeEnabled();
切勿使用
assert
语句或硬编码的
page.waitForTimeout()
。

Waiting Strategy

等待策略

typescript
// Wait for load state after navigation
await page.goto("/app");
await page.waitForLoadState("networkidle");

// Wait for specific element
await page.getByTestId("chat-intro").waitFor({ state: "visible", timeout: 10000 });

// Wait for URL change
await page.waitForFunction(() => window.location.href.includes("chatId="), null, { timeout: 10000 });

// Wait for network response
await page.waitForResponse(resp => resp.url().includes("/api/chat") && resp.status() === 200);
typescript
// 导航后等待加载状态
await page.goto("/app");
await page.waitForLoadState("networkidle");

// 等待特定元素
await page.getByTestId("chat-intro").waitFor({ state: "visible", timeout: 10000 });

// 等待URL变化
await page.waitForFunction(() => window.location.href.includes("chatId="), null, { timeout: 10000 });

// 等待网络响应
await page.waitForResponse(resp => resp.url().includes("/api/chat") && resp.status() === 200);

Best Practices

最佳实践

  1. Descriptive test names — clearly state expected behavior: 
    "should display greeting message when opening new chat"
  2. API-first setup — use 
    OnyxApiClient
    for backend state; reserve UI interactions for the behavior under test
  3. User isolation — tests that modify visible app state (sidebar, chat history) should run as the worker-specific user via 
    loginAsWorkerUser(page, testInfo.workerIndex)
    (not admin) and clean up resources in 
    afterAll
    . Each parallel worker gets its own user, preventing cross-contamination. Reserve 
    loginAsRandomUser
    for flows that require a brand-new user (e.g. onboarding)
  4. DRY helpers — extract reusable logic into 
    utils/
    with JSDoc comments
  5. No hardcoded waits — use 
    waitFor
    , 
    waitForLoadState
    , or web-first assertions
  6. Parallel-safe — no shared mutable state between tests. Prefer static, human-readable names (e.g. 
    "E2E-CMD Chat 1"
    ) and clean up resources by ID in 
    afterAll
    . This keeps screenshots deterministic and avoids needing to mask/hide dynamic text. Only fall back to timestamps (
    \
    test-${Date.now()}``) when resources cannot be reliably cleaned up or when name collisions across parallel workers would cause functional failures
  7. Error context — catch and re-throw with useful debug info (page text, URL, etc.)
  8. Tag slow tests — mark serial/slow tests with 
    @exclusive
    in the test title
  9. Visual regression — use 
    expectScreenshot()
    for UI consistency checks
  10. Minimal comments — only comment to clarify non-obvious intent; never restate what the next line of code does
  1. 描述性测试名称 — 清晰说明预期行为：例如
    "打开新聊天时应显示问候消息"
  2. 优先使用API设置 — 使用
    OnyxApiClient
    配置后端状态；仅在测试目标行为时使用UI交互
  3. 用户隔离 — 会修改可见应用状态（侧边栏、聊天历史）的测试应通过
    loginAsWorkerUser(page, testInfo.workerIndex)
    使用特定工作用户（而非管理员）运行，并在
    afterAll
    中清理资源。每个并行工作进程会获得独立用户，避免相互干扰。仅在需要全新用户的流程（例如引导）中使用
    loginAsRandomUser
  4. 提取可复用工具函数 — 将重复逻辑提取到
    utils/
    目录，并添加JSDoc注释
  5. 禁止硬编码等待 — 使用
    waitFor
    、
    waitForLoadState
    或Web优先断言
  6. 支持并行执行 — 测试间无共享可变状态。优先使用静态、易读的名称（例如
    "E2E-CMD Chat 1"
    ），并在
    afterAll
    中通过ID清理资源。这确保截图的确定性，避免需要屏蔽动态文本。仅当资源无法可靠清理或并行工作进程间名称冲突会导致功能失败时，才使用时间戳（
    \
    test-${Date.now()}``）
  7. 错误上下文 — 捕获异常并重新抛出，附带有用的调试信息（页面文本、URL等）
  8. 标记慢测试 — 在测试标题中使用
    @exclusive
    标记串行/慢测试
  9. 视觉回归测试 — 使用
    expectScreenshot()
    进行UI一致性检查
  10. 最小化注释 — 仅对非明显意图添加注释；切勿重复说明下一行代码的功能