hybrid-memory

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Hybrid Memory Skill

Hybrid Memory 技能

A self-contained hybrid memory system inspired by ZeroClaw's architecture. Combines vector embeddings (semantic search) with FTS5 keyword search (BM25 scoring) for powerful memory recall.

一款受ZeroClaw架构启发的独立混合记忆系统。结合向量嵌入（语义搜索）与FTS5关键词搜索（BM25评分），实现强大的记忆召回功能。

Purpose

用途

Give agents persistent memory with:

Semantic search - Find memories by meaning, not just keywords
Keyword search - Traditional BM25 text search via SQLite FTS5
Hybrid scoring - Weighted combination of vector and keyword similarity
Zero external dependencies - Pure SQLite + local embeddings

为Agent提供具备以下特性的持久化记忆：

语义搜索 - 通过含义而非仅关键词查找记忆
关键词搜索 - 借助SQLite FTS5实现传统BM25文本搜索
混合评分 - 向量与关键词相似度的加权组合
零外部依赖 - 纯SQLite + 本地嵌入向量

Architecture

架构

┌─────────────────────────────────────────────────────────────┐
│                        Hybrid Memory                          │
├─────────────────────────────────────────────────────────────┤
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐  │
│  │   Markdown   │  │   Chunker    │  │  Embedding API   │  │
│  │    Input     │ ─│>  (preserve  │─>│  (configurable)  │  │
│  │              │  │   headings)  │  │                  │  │
│  └──────────────┘  └──────────────┘  └──────────────────┘  │
│           │                                    │            │
│           ▼                                    ▼            │
│  ┌──────────────────────────────────────────────────────┐  │
│  │                 SQLite Database                       │  │
│  │  ┌─────────────┐  ┌─────────────┐  ┌───────────────┐ │  │
│  │  │ memories    │  │fts5 memories│  │embedding_cache│ │  │
│  │  │(id,content, │  │(content,    │  │(hash,vector)  │ │  │
│  │  │ vector_blob)│  │  metadata)   │  │               │ │  │
│  │  └─────────────┘  └─────────────┘  └───────────────┘ │  │
│  └──────────────────────────────────────────────────────┘  │
│                              │                              │
│                              ▼                              │
│  ┌──────────────────────────────────────────────────────┐  │
│  │              Hybrid Search Engine                     │  │
│  │         (cosine similarity + BM25 merge)              │  │
│  └──────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────┐
│                        Hybrid Memory                          │
├─────────────────────────────────────────────────────────────┤
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐  │
│  │   Markdown   │  │   Chunker    │  │  Embedding API   │  │
│  │    Input     │ ─│>  (preserve  │─>│  (configurable)  │  │
│  │              │  │   headings)  │  │                  │  │
│  └──────────────┘  └──────────────┘  └──────────────────┘  │
│           │                                    │            │
│           ▼                                    ▼            │
│  ┌──────────────────────────────────────────────────────┐  │
│  │                 SQLite Database                       │  │
│  │  ┌─────────────┐  ┌─────────────┐  ┌───────────────┐ │  │
│  │  │ memories    │  │fts5 memories│  │embedding_cache│ │  │
│  │  │(id,content, │  │(content,    │  │(hash,vector)  │ │  │
│  │  │ vector_blob)│  │  metadata)   │  │               │ │  │
│  │  └─────────────┘  └─────────────┘  └───────────────┘ │  │
│  └──────────────────────────────────────────────────────┘  │
│                              │                              │
│                              ▼                              │
│  ┌──────────────────────────────────────────────────────┐  │
│  │              Hybrid Search Engine                     │  │
│  │         (cosine similarity + BM25 merge)              │  │
│  └──────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────┘

Setup

安装配置

bash

cd /job/.pi/skills/hybrid-memory
npm install

bash

cd /job/.pi/skills/hybrid-memory
npm install

Configuration

配置项

Set in your environment or

.env

bash

undefined

在环境变量或

.env

文件中设置：

bash

undefined

Required: OpenAI API key for embeddings (or use local embedding model)

必填：用于嵌入向量的OpenAI API密钥（或使用本地嵌入模型）

OPENAI_API_KEY=sk-...

Optional: Embedding model (default: text-embedding-3-small)

可选：嵌入模型（默认值：text-embedding-3-small）

EMBEDDING_MODEL=text-embedding-3-small

Optional: Vector weight in hybrid scoring (0-1, default: 0.7)

可选：混合评分中的向量权重（0-1，默认值：0.7）

HYBRID_VECTOR_WEIGHT=0.7

Optional: Database path (default: /job/data/hybrid-memory.db)

可选：数据库路径（默认值：/job/data/hybrid-memory.db）

HYBRID_MEMORY_DB_PATH=/job/data/hybrid-memory.db

undefined

HYBRID_MEMORY_DB_PATH=/job/data/hybrid-memory.db

undefined

Commands

命令

Initialize Database

初始化数据库

bash

memory-init

Creates SQLite database with:

```
memories
```
table (id, content, vector_blob, metadata, created_at)
```
memories_fts
```
FTS5 virtual table for full-text search
```
embedding_cache
```
table for LRU caching

bash

memory-init

创建包含以下内容的SQLite数据库：

```
memories
```
表（id, content, vector_blob, metadata, created_at）
```
memories_fts
```
用于全文搜索的FTS5虚拟表
```
embedding_cache
```
用于LRU缓存的表

Store Memory

存储记忆

bash

memory-store "Your memory content here" --tags project,meeting
memory-store -f /path/to/file.md --source "Documentation"

Options:

```
--tags
```
- Comma-separated tags for filtering
```
--source
```
- Source attribution
```
--id
```
- Custom memory ID

bash

memory-store "Your memory content here" --tags project,meeting
memory-store -f /path/to/file.md --source "Documentation"

可选参数：

```
--tags
```
- 用于过滤的逗号分隔标签
```
--source
```
- 来源标注
```
--id
```
- 自定义记忆ID

Search Memories

搜索记忆

bash

undefined

bash

undefined

Semantic search (vector only)

语义搜索（仅向量）

memory-search "authentication middleware"

Keyword search (BM25)

关键词搜索（BM25）

memory-search "authentication middleware" --mode keyword

Hybrid search (default)

混合搜索（默认）

memory-search "authentication middleware" --mode hybrid

With filters

带过滤条件

memory-search "deployment" --tags production --limit 10

undefined

memory-search "deployment" --tags production --limit 10

undefined

Recall (Semantic + Contextual)

记忆召回（语义+上下文）

bash

undefined

bash

undefined

Find most relevant memories for current context

查找与当前上下文最相关的记忆

memory-recall "How do I configure the database?"

Top-K with threshold

带阈值的Top-K召回

memory-recall "error handling" --top-k 5 --threshold 0.7

undefined

memory-recall "error handling" --top-k 5 --threshold 0.7

undefined

Memory Management

记忆管理

bash

undefined

bash

undefined

List recent memories

列出近期记忆

memory-list --limit 20

Delete memory

删除记忆

memory-delete <memory-id>

Export memories

导出记忆

memory-export --format json > memories.json memory-export --format markdown > memories.md

Get stats

获取统计信息

memory-stats

undefined

memory-stats

undefined

Tools Added

新增工具

When this skill is active, the agent gains access to:

启用该技能后，Agent将获得以下工具的访问权限：

memory_recall

memory_recall

Recall relevant memories based on query context.

javascript

// Use in agent prompt for contextual memory
memory_recall({
  query: "How did I implement authentication?",
  top_k: 5,
  threshold: 0.6
})

根据查询上下文召回相关记忆。

javascript

// 在Agent提示词中用于上下文记忆
memory_recall({
  query: "How did I implement authentication?",
  top_k: 5,
  threshold: 0.6
})

memory_store

memory_store

Store new memories with automatic embedding.

javascript

memory_store({
  content: "User prefers TypeScript over JavaScript",
  tags: ["preferences", "user-profile"],
  source: "conversation"
})

自动生成嵌入向量并存储新记忆。

javascript

memory_store({
  content: "User prefers TypeScript over JavaScript",
  tags: ["preferences", "user-profile"],
  source: "conversation"
})

memory_search

memory_search

Search stored memories.

javascript

memory_search({
  query: "database configuration",
  mode: "hybrid",  // "vector", "keyword", "hybrid"
  limit: 10
})

搜索已存储的记忆。

javascript

memory_search({
  query: "database configuration",
  mode: "hybrid",  // "vector", "keyword", "hybrid"
  limit: 10
})

Usage in Agent Prompt

在Agent提示词中的使用方法

When this skill is active, include this context:

undefined

启用该技能后，需在提示词中加入以下上下文：

undefined

Memory System

记忆系统

You have access to a hybrid memory system (vector + keyword search) via:

memory_recall(query, top_k?, threshold?)

- Recall relevant context

```
memory_store(content, tags?, source?)
```
- Store new memories
```
memory_search(query, mode?, limit?)
```
- Search memories

Use memory_recall() to:

Remember previous conversations
Find relevant code patterns
Access stored documentation
Maintain context across sessions

Use memory_store() to:

Save user preferences
Store discovered patterns
Remember decisions made
Document important findings

The hybrid search combines:

Semantic similarity (70% weight): Finds by meaning
BM25 keyword scoring (30% weight): Finds by exact words

undefined

你可通过以下方式访问混合记忆系统（向量+关键词搜索）：

memory_recall(query, top_k?, threshold?)

- 召回相关上下文

```
memory_store(content, tags?, source?)
```
- 存储新记忆
```
memory_search(query, mode?, limit?)
```
- 搜索记忆

使用memory_recall()来：

记住之前的对话内容
查找相关代码模式
访问已存储的文档
在会话间维持上下文

使用memory_store()来：

保存用户偏好
存储发现的模式
记住已做出的决策
记录重要发现

混合搜索结合了：

语义相似度（70%权重）：通过含义查找
BM25关键词评分（30%权重）：通过精确匹配词查找

undefined

Technical Details

技术细节

Embedding Strategy

嵌入策略

Chunking: Paragraph-level with heading preservation
Model: OpenAI text-embedding-3-small (1536 dimensions)
Normalization: L2-normalized for cosine similarity
Caching: Hash-based embedding cache (SHA-256 of content)

文本分块：保留标题的段落级分块
模型：OpenAI text-embedding-3-small（1536维度）
归一化：L2归一化用于余弦相似度计算
缓存：基于哈希的嵌入向量缓存（内容的SHA-256哈希值）

Hybrid Scoring

混合评分

final_score = (vector_weight * vector_score) + 
              (keyword_weight * bm25_score_normalized)

Example weights:
- vector_weight = 0.7 (semantic meaning)
- keyword_weight = 0.3 (exact word matches)

final_score = (vector_weight * vector_score) + 
              (keyword_weight * bm25_score_normalized)

示例权重：
- vector_weight = 0.7（语义含义）
- keyword_weight = 0.3（精确词匹配）

Similarity Search

相似度搜索

Vector similarity uses cosine distance on normalized embeddings:

sql

SELECT id, content, 
       (vector_dot_product(embedding, :query_vec)) as similarity
FROM memories
ORDER BY similarity DESC
LIMIT :limit

向量相似度基于归一化嵌入向量的余弦距离：

sql

SELECT id, content, 
       (vector_dot_product(embedding, :query_vec)) as similarity
FROM memories
ORDER BY similarity DESC
LIMIT :limit

FTS5 BM25 Scoring

FTS5 BM25评分

sql

SELECT rowid, bm25(memories_fts, 1.2, 0.75) as score
FROM memories_fts
WHERE memories_fts MATCH :query
ORDER BY score DESC

sql

SELECT rowid, bm25(memories_fts, 1.2, 0.75) as score
FROM memories_fts
WHERE memories_fts MATCH :query
ORDER BY score DESC

File Structure

文件结构

.pi/skills/hybrid-memory/
├── SKILL.md                 # This file
├── package.json
├── lib/
│   ├── db.js               # SQLite connection + schema
│   ├── embeddings.js       # OpenAI embedding client
│   ├── chunker.js          # Text chunking logic
│   ├── search.js           # Vector + keyword + hybrid search
│   └── cache.js            # Embedding cache management
├── bin/
│   ├── memory-init.js      # Initialize database
│   ├── memory-store.js     # Store memories CLI
│   ├── memory-search.js    # Search memories CLI
│   ├── memory-recall.js    # Recall with context
│   ├── memory-list.js      # List memories
│   ├── memory-delete.js    # Delete memory
│   ├── memory-export.js    # Export to various formats
│   └── memory-stats.js     # Database statistics
└── tests/
    └── memory.test.js      # Test suite

.pi/skills/hybrid-memory/
├── SKILL.md                 # 本文档
├── package.json
├── lib/
│   ├── db.js               # SQLite连接 + 数据库架构
│   ├── embeddings.js       # OpenAI嵌入客户端
│   ├── chunker.js          # 文本分块逻辑
│   ├── search.js           # 向量+关键词+混合搜索
│   └── cache.js            # 嵌入向量缓存管理
├── bin/
│   ├── memory-init.js      # 初始化数据库
│   ├── memory-store.js     # 记忆存储CLI
│   ├── memory-search.js    # 记忆搜索CLI
│   ├── memory-recall.js    # 上下文召回
│   ├── memory-list.js      # 列出记忆
│   ├── memory-delete.js    # 删除记忆
│   ├── memory-export.js    # 导出为多种格式
│   └── memory-stats.js     # 数据库统计信息
└── tests/
    └── memory.test.js      # 测试套件

Performance Characteristics

性能特性

Metric	Expected
Embedding latency	~100-300ms (OpenAI API)
Vector search	<10ms for 10k memories
Keyword search	<50ms for 10k memories
Storage per memory	~6KB (1536 dims * 4 bytes)
Database size	~60MB for 10k memories

指标	预期值
嵌入向量延迟	~100-300ms（OpenAI API）
向量搜索	1万条记忆时耗时<10ms
关键词搜索	1万条记忆时耗时<50ms
单条记忆存储大小	~6KB（1536维度 * 4字节）
数据库大小	1万条记忆时约60MB

When to Use

适用场景

Long-running conversations - Maintain context across sessions
Code knowledge bases - Remember patterns and decisions
Documentation search - Semantic doc retrieval
User preferences - Remember how users like things done
Research accumulation - Build knowledge over time

长对话场景 - 在会话间维持上下文
代码知识库 - 记住代码模式与决策
文档搜索 - 语义化文档检索
用户偏好记录 - 记住用户的使用习惯
研究积累 - 逐步构建知识体系

Integration Example

集成示例

javascript

// In agent workflow:

// 1. Before answering, recall relevant context
const relevant = memory_recall({
  query: userQuestion,
  top_k: 3
});

// 2. Include context in LLM prompt
const prompt = `
  Previous relevant context:
  ${relevant.map(m =>m.content).join('\n---\n')}
  
  User question: ${userQuestion}
`;

// 3. After answering, store the exchange
memory_store({
  content: `Q: ${userQuestion}\nA: ${answer}`,
  tags: ["conversation", "q-and-a"],
  source: "agent-session"
});

javascript

// 在Agent工作流中：

// 1. 回答前召回相关上下文
const relevant = memory_recall({
  query: userQuestion,
  top_k: 3
});

// 2. 将上下文加入LLM提示词
const prompt = `
  Previous relevant context:
  ${relevant.map(m =>m.content).join('\n---\n')}
  
  User question: ${userQuestion}
`;

// 3. 回答后存储对话内容
memory_store({
  content: `Q: ${userQuestion}\nA: ${answer}`,
  tags: ["conversation", "q-and-a"],
  source: "agent-session"
});

Inspiration

灵感来源

This skill is inspired by ZeroClaw's memory architecture:

"Full-Stack Memory System" - Zero overhead, custom implementation
"Hybrid Merge" - Weighted combination of vector + keyword
"Safe Reindex" - Atomic rebuilds with no downtime

本技能受ZeroClaw的记忆架构启发：

"Full-Stack Memory System" - 零额外开销的自定义实现
"Hybrid Merge" - 向量与关键词的加权组合
"Safe Reindex" - 无停机的原子重建