google-cloud-solution-hybrid-search-alloydb

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Dynamic Hybrid Search using AlloyDB

基于AlloyDB的动态混合搜索

This skill provides a workflow to design and implement secure, low-latency, and high-accuracy hybrid search solutions combining structured dataset filtering, vector search indexing, faceted metadata filtering, semantic reranking, recall evaluation, in-database AI validation, database abstraction layers, and serverless application hosting.

本技能提供一套工作流，用于设计和实现安全、低延迟、高精度的混合搜索解决方案，涵盖结构化数据集过滤、向量搜索索引、分面元数据过滤、语义重排序、召回率评估、数据库内AI验证、数据库抽象层以及无服务器应用托管。

Overview of the workflow

工作流概述

The workflow consists of the following phases:

Requirements discovery. Gather detailed requirements related to the cloud workload or use case that the user needs assistance for.
Solution architecture. Use the requirements that were gathered in Phase 1 to generate a detailed solution architecture for the cloud workload or use case.
Solution validation. Create a plan to validate the generated solution, generate validation instructions and scripts, and run the validation.
Solution packaging and presentation. Consolidate the generated content and present the solution.

Important notes about the workflow:

Strict phase separation: During Phase 1 (Requirements discovery), when you ask the user clarifying questions, DON'T recommend, propose, or outline any architectural designs, cloud services, or component mappings. This prevents premature architecture commitments or hallucinations before the full scope is understood.
Halting for approval: For any step where you are instructed to "obtain approval before proceeding", you MUST stop executing, present the completed tasks to the user, and wait for their explicit approval. You MUST NOT proceed to execute any subsequent tasks or generate any further guidance in that response.
Ground all generated content: For all tasks across all phases, you MUST first look in the following resources:
- Product Mapping,
- Design Recommendations for the required guidance. If the guidance does not provide the required information, you MUST ground the generated content by using the following resources:
- Google Developer Knowledge MCP server: https://developers.google.com/knowledge/mcp.md.txt
  - Server: https://developerknowledge.googleapis.com/mcp
    - Tools:
      - developerknowledge:search_documents
      - developerknowledge:get_documents
      - developerknowledge:answer_query
- Relevant skills from https://github.com/google/skills
- Official Google Cloud documentation in Related Guidance

工作流包含以下阶段：

需求发现：收集用户需要协助的云工作负载或用例的详细需求。
解决方案架构：利用阶段1收集的需求，为云工作负载或用例生成详细的解决方案架构。
解决方案验证：创建计划验证生成的解决方案，生成验证说明和脚本并执行验证。
解决方案打包与展示：整合生成的内容并展示解决方案。

工作流重要说明：

严格的阶段分离：在阶段1（需求发现）中，当你向用户询问澄清问题时，请勿推荐、提议或概述任何架构设计、云服务或组件映射。这可防止在完全理解范围之前做出过早的架构承诺或产生幻觉内容。
等待批准后再继续：对于任何要求“获得批准后再继续”的步骤，你必须停止执行，向用户展示已完成的任务，并等待其明确批准。在该回复中，你不得继续执行任何后续任务或生成更多指导内容。
所有生成内容需有依据：对于所有阶段的所有任务，你必须首先参考以下资源：
- 产品映射，
- 设计建议获取所需指导。如果这些指导无法提供所需信息，你必须通过以下资源为生成的内容提供依据：
- Google Developer Knowledge MCP服务器： https://developers.google.com/knowledge/mcp.md.txt
  - 服务器地址：https://developerknowledge.googleapis.com/mcp
    - 工具：
      - developerknowledge:search_documents
      - developerknowledge:get_documents
      - developerknowledge:answer_query
- https://github.com/google/skills中的相关技能
- 相关指导中的官方Google Cloud文档

Product Renaming & Terminology

产品重命名与术语规范

When generating solution designs, architecture diagrams, and documentation, check the latest Google Cloud documentation for the most up-to-date product names. The table below provides examples of name mappings to be aware of. Note that underlying APIs, Terraform resources, and IAM roles may retain their legacy identifiers.

<table> <thead> <tr> <th>Legacy Name</th> <th>Updated Name</th> <th>Notes</th> </tr> </thead> <tbody> <tr> <td>Vertex AI</td> <td>Gemini Enterprise Agent Platform</td> <td>Gemini Enterprise Agent Platform can be shortened to Agent Platform after first instance</td> </tr> <tr> <td>Vertex AI Embedding</td> <td>Text embedding on Gemini Enterprise Agent Platform</td> <td>This refers to the text embedding models available on Gemini Enterprise Agent Platform</td> </tr> <tr> <td>Vertex AI Matching Engine</td> <td>Vector Search</td> <td></td> </tr> </tbody> </table>

在生成解决方案设计、架构图和文档时，请查阅最新的Google Cloud文档以获取最新的产品名称。下表提供了需要注意的名称映射示例。请注意，底层API、Terraform资源和IAM角色可能仍保留其旧标识符。

<table> <thead> <tr> <th>旧名称</th> <th>更新后名称</th> <th>说明</th> </tr> </thead> <tbody> <tr> <td>Vertex AI</td> <td>Gemini Enterprise Agent Platform</td> <td>Gemini Enterprise Agent Platform首次提及后可简称为Agent Platform</td> </tr> <tr> <td>Vertex AI Embedding</td> <td>Text embedding on Gemini Enterprise Agent Platform</td> <td>指Gemini Enterprise Agent Platform上可用的文本嵌入模型</td> </tr> <tr> <td>Vertex AI Matching Engine</td> <td>Vector Search</td> <td></td> </tr> </tbody> </table>

Phase 1: Requirements discovery

阶段1：需求发现

In this phase, you must gather detailed requirements related to the hybrid search workload that the user wants to design and deploy in Google Cloud.

Acknowledge provided requirements: If the user's prompt already contains some requirements (functional or non-functional, such as catalog size, search modalities, faceted attributes, or latency targets), you MUST explicitly acknowledge and restate all of these requirements in your response. Do NOT ask the user to describe or re-describe any requirements that they have already provided in the prompt.

Complete the following steps strictly in the specified order:

Step 1: Ask the user to describe the functional requirements of the workload, including catalog dataset details (e.g., e-commerce apparel, retail products, patent database), search modalities (natural language text, visual search, attribute filters), metadata attributes for faceted filtering (e.g.,
```
category
```
,
```
sub_category
```
,
```
color
```
,
```
gender
```
,
```
price
```
), and quality checks (reranking, LLM validation).
Step 2: You MUST explicitly ask the user to describe ALL of the following six categories of non-functional requirements. You need this information because each category represents a critical architectural pillar, and neglecting any of them can result in a solution that is insecure, unreliable, or inefficient (do NOT omit any of them):
- Security, privacy, and compliance: E.g., private VPC endpoints, Private Service Connect, Direct VPC Egress, and access control.
- Reliability: E.g., high availability, failover, disaster recovery goals (RTO/RPO), regional vs multi-region AlloyDB topology.
- Cost: E.g., budget constraints for compute, database instances, and Gemini Enterprise Agent Platform API calls.
- Operational excellence: E.g., monitoring, logging, dashboards, and automated deployment.
- Performance: E.g., target P95 query latency (e.g., < 100ms), vector search recall target (e.g., > 95%), catalog item scale, and QPS expectations.
- Sustainability: E.g., carbon footprint, low-carbon region selection.
Step 3: Ask the user whether the workload currently runs on other cloud providers or on-premises.
- If the user's answer is "yes", then ask the user to describe the architecture of the current deployment.
- If the user's answer is "no", then proceed to the next step.
Step 4: Ask the user to describe dependencies, if any, on other workloads, products, or tools (e.g., existing inventory databases, ERP systems, application runtime languages like Java or Python).
Step 5: Review the input that the user has provided so far, and check whether there are any ambiguities, conflicts, or contradictions in the functional requirements, non-functional requirements, and dependencies. You MUST compare all requirements against each other to identify any conflicts.

If you identify any ambiguities, conflicts, or contradictions in the requirements that the user has provided, you MUST do the following for each ambiguity, conflict, or contradiction:
- Identify exactly where each contradiction lies and explain to the user why the requirements are incompatible and cannot be simultaneously satisfied. Do NOT treat fundamental contradictions as design choice questions (e.g., asking how to implement or configure a conflicting requirement).
- Ask the user to clarify their trade-off preferences to resolve the contradiction.
- If the user delegates the choice to you (e.g., the user replies with "do what you think is best" or "you decide"), then provide a clear suggestion to resolve the ambiguity or contradiction, explain your reasoning, and ask the user to approve your suggestion.
Critical: Until all the ambiguities and contradictions that you identify are resolved according to the preceding guidance, you must NOT recommend or generate any architecture design or Google Cloud product recommendations.
Step 6: Summarize the functional and non-functional requirements provided by the user into a consolidated requirements summary.
Step 7: Present the generated requirements summary to the user and obtain approval (the user MUST explicitly say "yes" or "I approve") before proceeding to Phase 2.

Important: STOP, DON'T proceed to generate architecture diagram, architecture description or product recommendations until you have confirmed the generated requirements summary and resolved all ambiguities and contradictions in this phase.

在本阶段，你必须收集用户希望在Google Cloud中设计和部署的混合搜索工作负载的详细需求。

确认已提供的需求：如果用户的提示中已包含一些需求（功能性或非功能性，例如目录规模、搜索模式、分面属性或延迟目标），你必须在回复中明确确认并重述所有这些需求。请勿要求用户描述或重新描述他们已在提示中提供的任何需求。

严格按照指定顺序完成以下步骤：

步骤1：请用户描述工作负载的功能性需求，包括目录数据集详情（如电商服装、零售产品、专利数据库）、搜索模式（自然语言文本、视觉搜索、属性过滤）、用于分面过滤的元数据属性（如
```
category
```
、
```
sub_category
```
、
```
color
```
、
```
gender
```
、
```
price
```
）以及质量检查（重排序、LLM验证）。
步骤2：你必须明确请用户描述以下所有六大类非功能性需求。你需要这些信息，因为每一类都代表一个关键的架构支柱，忽略任何一类都可能导致解决方案不安全、不可靠或效率低下（请勿省略任何一类）：
- 安全、隐私与合规：例如，私有VPC端点、Private Service Connect、Direct VPC Egress以及访问控制。
- 可靠性：例如，高可用性、故障转移、灾难恢复目标（RTO/RPO）、AlloyDB的单区域 vs 多区域拓扑。
- 成本：例如，计算、数据库实例和Gemini Enterprise Agent Platform API调用的预算限制。
- 运维卓越性：例如，监控、日志记录、仪表板和自动化部署。
- 性能：例如，目标P95查询延迟（如< 100ms）、向量搜索召回率目标（如> 95%）、目录条目规模和QPS预期。
- 可持续性：例如，碳足迹、低碳区域选择。
步骤3：询问用户该工作负载当前是否在其他云提供商或本地环境运行。
- 如果用户回答“是”，则请用户描述当前部署的架构。
- 如果用户回答“否”，则继续下一步。
步骤4：询问用户是否依赖其他工作负载、产品或工具（如现有库存数据库、ERP系统、Java或Python等应用运行时语言）。
步骤5：回顾用户到目前为止提供的输入，检查功能性需求、非功能性需求和依赖项中是否存在任何歧义、冲突或矛盾。你必须比较所有需求以识别任何冲突。

如果你在用户提供的需求中发现任何歧义、冲突或矛盾，你必须为每个歧义、冲突或矛盾执行以下操作：
- 准确指出每个矛盾所在，并向用户解释为什么这些需求不兼容且无法同时满足。请勿将根本性矛盾视为设计选择问题（例如，询问如何实现或配置冲突的需求）。
- 请用户明确其权衡偏好以解决矛盾。
- 如果用户将选择权交给你（例如，用户回复“你认为最好的方式来做”或“你来决定”），则提供明确的解决方案建议，解释你的推理，并请用户批准你的建议。
关键提示：在你识别的所有歧义和矛盾按照上述指导解决之前，你不得推荐或生成任何架构设计或Google Cloud产品建议。
步骤6：将用户提供的功能性和非功能性需求汇总为一份整合的需求摘要。
步骤7：向用户展示生成的需求摘要，并在进入阶段2前获得批准（用户必须明确说“是”或“我批准”）。

重要提示：停止，在确认生成的需求摘要并解决本阶段的所有歧义与矛盾之前，请勿继续生成架构图、架构描述或产品建议。

Phase 2: Solution architecture

阶段2：解决方案架构

Task 2.1: Identify Google Cloud products and features required for the workload.

任务2.1：确定工作负载所需的Google Cloud产品和功能。

Step 1: Recommend products and features that are appropriate for each component of the user's workload, prioritizing Google Cloud products.

Important: The Google Cloud products and features that you recommend MUST be consistent with the guidance in Product Mapping.
Step 4: Present the generated product recommendations to the user and obtain approval (the user MUST explicitly say "yes" or "I approve") before proceeding to Task 2.2.

Important: STOP, DON'T proceed to generate architecture diagram until you have confirmed the generated product recommendations with the user.

步骤1：为用户工作负载的每个组件推荐合适的产品和功能，优先选择Google Cloud产品。

重要提示：你推荐的Google Cloud产品和功能必须与产品映射中的指导一致。
步骤4：向用户展示生成的产品建议，并在进入任务2.2前获得批准（用户必须明确说“是”或“我批准”）。

重要提示：停止，在与用户确认生成的产品建议之前，请勿继续生成架构图。

Task 2.2: Generate an architecture diagram and description

任务2.2：生成架构图和描述

Step 1: Generate an architecture diagram in the Mermaid format: https://github.com/mermaid-js/mermaid.

The diagram must show the data flows and request flows across the components of the architecture, based on the gathered requirements and product recommendations. The diagram MUST explicitly show both the ingestion pipeline and serving pipeline.

The following is an example of the data flows and request flows that the architecture diagram should show:
- Ingestion pipeline: Catalog Data -> AlloyDB Table (
```
apparels
```
  ) -> B-Tree Indexes on Facets -> Text embedding (
```
text-embedding-005
```
  ) -> ScaNN Vector Index.
- Serving pipeline: User Browser -> Cloud Run Web App -> MCP Toolbox for Databases -> AlloyDB Single-Query Hybrid Search (ScaNN Vector Search + SQL WHERE Filters) ->
```
ai.rank
```
  Reranker -> Gemini Pro
```
ai.generate
```
  Quality Validation -> Validated Results -> User Browser.
Step 2: Generate a description that explains the purpose of each component, the relationships between the components, and the task flow or data flow.
Step 3: Present the generated architecture diagram and description to the user and obtain approval (the user MUST explicitly say "yes" or "I approve") before proceeding to Task 2.3.

Important: STOP, DON'T proceed to generate design recommendations until you have confirmed the generated architecture description with the user.

步骤1：以Mermaid格式生成架构图： https://github.com/mermaid-js/mermaid.

该图必须根据收集的需求和产品建议，展示架构各组件之间的数据流和请求流。图中必须明确展示摄入管道和服务管道。

以下是架构图应展示的数据流和请求流示例：
- 摄入管道：目录数据 -> AlloyDB表 (
```
apparels
```
  ) -> 分面字段的B-Tree索引 -> Text embedding (
```
text-embedding-005
```
  ) -> ScaNN向量索引。
- 服务管道：用户浏览器 -> Cloud Run Web应用 -> MCP Toolbox for Databases -> AlloyDB单查询混合搜索（ScaNN向量搜索 + SQL WHERE过滤） ->
```
ai.rank
```
  重排序器 -> Gemini Pro
```
ai.generate
```
  质量验证 -> 验证后的结果 -> 用户浏览器。
步骤2：生成描述，解释每个组件的用途、组件之间的关系以及任务流或数据流。
步骤3：向用户展示生成的架构图和描述，并在进入任务2.3前获得批准（用户必须明确说“是”或“我批准”）。

重要提示：停止，在与用户确认生成的架构描述之前，请勿继续生成设计建议。

Task 2.3: Generate design recommendations.

任务2.3：生成设计建议。

Step 1: Generate design recommendations and best practices to optimally configure each component in the architecture based on the workload requirements.

Important:
- When you generate design recommendations, consider the following:
  - Functional requirements that were gathered in Phase 1.
  - Non-functional requirements that were gathered in Phase 1.
- Align the generated design recommendations with the recommendations in Design Recommendations.
- To generate guidance for the non-functional requirements, use the following skills:
  - ```
  google-cloud-waf-security
```
- ```
google-cloud-waf-reliability
```
  - ```
  google-cloud-waf-cost-optimization
```
- ```
google-cloud-waf-operational-excellence
```
  - ```
  google-cloud-waf-performance-optimization
```
- ```
google-cloud-waf-sustainability
```
Step 2: Present the generated recommendations to the user and obtain approval (the user MUST explicitly say "yes" or "I approve") before proceeding to Task 2.4.

Important: STOP, DON'T proceed to generate deployment guidance until you have confirmed the design recommendations with the user.

步骤1：根据工作负载需求，为架构中的每个组件生成优化配置的设计建议和最佳实践。

重要提示：
- 生成设计建议时，请考虑以下因素：
  - 阶段1收集的功能性需求。
  - 阶段1收集的非功能性需求。
- 生成的设计建议需与设计建议中的指导保持一致。
- 为生成非功能性需求的指导，请使用以下技能：
  - ```
  google-cloud-waf-security
```
- ```
google-cloud-waf-reliability
```
  - ```
  google-cloud-waf-cost-optimization
```
- ```
google-cloud-waf-operational-excellence
```
  - ```
  google-cloud-waf-performance-optimization
```
- ```
google-cloud-waf-sustainability
```
步骤2：向用户展示生成的建议，并在进入任务2.4前获得批准（用户必须明确说“是”或“我批准”）。

重要提示：停止，在与用户确认设计建议之前，请勿继续生成部署指南。

Task 2.4: Generate deployment guidance.

任务2.4：生成部署指南。

Step 1: Generate guidance to deploy the solution, including the following:
- AlloyDB DDL & SQL setup scripts for extensions (
```
google_ml_integration
```
  ,
```
alloydb_scan
```
  ), tables, B-Tree indexes, ScaNN vector indexes, hybrid search SQL, and Gemini validation CTEs.
- MCP Toolbox deployment configuration on Cloud Run.
- Python Cloud Run Function shim deployment command.
- Application deployment command (
```
gcloud run deploy {app_name}
```
  ).
- Terraform code or
```
gcloud
```
  CLI commands to create required infrastructure.
Important: The deployment guidance that you generate MUST be consistent with the guidance in the following resources:
- Related Guidance
- Relevant skills in https://github.com/google/skills/tree/main/skills/cloud
Step 2: Present the generated deployment guidance to the user and obtain approval (the user MUST explicitly say "yes" or "I approve") before proceeding to Phase 3.

Important: STOP, DON'T proceed to generate solution validation until you have confirmed the deployment guidance with the user.

步骤1：生成解决方案的部署指南，包括以下内容：
- 用于扩展（
```
google_ml_integration
```
  、
```
alloydb_scan
```
  ）、表、B-Tree索引、ScaNN向量索引、混合搜索SQL和Gemini验证CTE的AlloyDB DDL和SQL设置脚本。
- Cloud Run上的MCP Toolbox部署配置。
- Python Cloud Run Function垫片部署命令。
- 应用部署命令（
```
gcloud run deploy {app_name}
```
  ）。
- 创建所需基础设施的Terraform代码或
```
gcloud
```
  CLI命令。
重要提示：你生成的部署指南必须与以下资源中的指导一致：
- 相关指导
- https://github.com/google/skills/tree/main/skills/cloud中的相关技能
步骤2：向用户展示生成的部署指南，并在进入阶段3前获得批准（用户必须明确说“是”或“我批准”）。

重要提示：停止，在与用户确认部署指南之前，请勿继续生成解决方案验证内容。

Phase 3: Solution validation

阶段3：解决方案验证

Task 3.1: Pre-deployment validation

任务3.1：部署前验证

Step 1: Create a pre-deployment plan to statically validate the generated solution and verify that it meets the workload requirements without provisioning live resources:
- Deployment dry-run: Validate infrastructure syntax and preview the resources that will be provisioned using dry-run commands (e.g.,
```
terraform plan
```
  or (where supported)
```
gcloud ... --dry-run
```
  ).
- Architecture & policy analysis: Perform static verification of network routing topologies, firewall rules, and IAM enforcement against best practices.
Step 2: Present the static validation plan to the user, obtain approval (the user MUST explicitly say "yes" or "I approve"), and execute the dry-run commands.
Step 3: Troubleshoot and fix any errors or policy discrepancies identified during dry-run checks until validation succeeds.
Step 4: Proceed to Task 3.2

步骤1：创建部署前计划，静态验证生成的解决方案，无需配置实时资源即可验证其是否满足工作负载需求：
- 部署预演：使用预演命令（如
```
terraform plan
```
  或（支持的情况下）
```
gcloud ... --dry-run
```
  ）验证基础设施语法并预览将配置的资源。
- 架构与策略分析：针对最佳实践，对网络路由拓扑、防火墙规则和IAM实施进行静态验证。
步骤2：向用户展示静态验证计划，获得批准（用户必须明确说“是”或“我批准”），并执行预演命令。
步骤3：排查并修复预演检查中发现的任何错误或策略差异，直到验证成功。
步骤4：进入任务3.2

Task 3.2: Runtime validation (Post-deployment)

任务3.2：运行时验证（部署后）

Step 1: Ask the user whether they choose to deploy the infrastructure now to perform live runtime verification, or skip directly to Phase 4.
Step 2: If the user chooses to deploy the infrastructure:
- After the user deploys the infrastructure, generate runtime verification commands (using tools like
```
curl
```
  ,
```
ping
```
  , or
```
gcloud
```
  ) and provide them to the user to execute, to test live endpoint reachability, networking paths, and load balancer routing.
- Troubleshoot any deployment or runtime routing issues until checks pass.
Step 3: Proceed to Phase 4.

步骤1：询问用户是否选择现在部署基础设施以执行实时运行时验证，或直接跳至阶段4。
步骤2：如果用户选择部署基础设施：
- 用户部署基础设施后，生成运行时验证命令（使用
```
curl
```
  、
```
ping
```
  或
```
gcloud
```
  等工具）并提供给用户执行，以测试实时端点可达性、网络路径和负载均衡器路由。
- 排查任何部署或运行时路由问题，直到检查通过。
步骤3：进入阶段4。

Phase 4: Solution packaging and presentation

阶段4：解决方案打包与展示

Step 1: Consolidate the final text artifacts that were generated in Phase 2 into a single Markdown file named
```
solution-architecture-guide.md
```
, based on the template in Output Template.
Step 2: Request the user's permission to write the code files in the user's workspace.
Step 3: After the user gives permission, write the final code files in the user's workspace.

步骤1：将阶段2中生成的最终文本工件整合到一个名为
```
solution-architecture-guide.md
```
的Markdown文件中，基于输出模板中的模板。
步骤2：请求用户允许在其工作区中写入代码文件。
步骤3：获得用户许可后，在其工作区中写入最终代码文件。