Personal Wiki — Ingest
Process raw source documents into structured, interlinked wiki pages.
Identify Sources to Process
Determine which files need ingestion:
- If the user specifies a file or files, use those
- If the user says "process new sources", "ingest", or similar without specifying files — run batch detection:
- List all files in (excluding )
- Read and extract all previously ingested source filenames from entries
- Any file in not listed in the log is unprocessed
- Show the user the list of unprocessed files found
- If no unprocessed files are found, tell the user
Process Each Source
For each source file, process autonomously — read, create pages, and report results. No confirmation step between sources.
1. Read the source completely
Read the entire file. If the file contains image references, note them — read the images separately if they contain important information.
For books: support both chapter-by-chapter and whole-book ingestion. When ingesting a chapter, create or update a parent book page in
that links to all chapter summaries. When ingesting a whole book, create a single comprehensive source page.
2. Create source summary page
Create a new file in
named after the source (slugified). Include:
---
tags: [relevant, tags]
sources: [original-filename.md]
created: YYYY-MM-DD
updated: YYYY-MM-DD
---
# Source Title
**Source:** original-filename.md
**Date ingested:** YYYY-MM-DD
**Type:** article | paper | transcript | notes | book | chapter | etc.
## Summary
Structured summary of the source content.
## Key Claims
- Claim 1
- Claim 2
- ...
## Entities Mentioned
- [[Entity Name]] — brief context
- ...
## Concepts Covered
- [[Concept Name]] — brief context
- ...
3. Update entity and concept pages
For each entity (person, organization, product, tool) and concept (idea, framework, theory, pattern) mentioned in the source:
If a wiki page already exists:
- Read the existing page
- Add new information from this source
- Add the source to the frontmatter list
- Update the date
- Flag any contradictions with existing content using a callout block:
> [!warning] Contradiction
> Source A claims X, but Source B claims Y.
> Sources: [[Source A]], [[Source B]]
If no wiki page exists and the topic is substantive enough:
- Create a new page in the appropriate subdirectory:
- for people, organizations, products, tools
- for ideas, frameworks, theories, patterns
- Include YAML frontmatter with tags, sources, created, and updated fields
- Write a focused summary based on what this source says about the topic
If the topic is only mentioned in passing:
- Use a without creating a page — the lint pass will flag frequently-mentioned-but-missing pages later
4. Add wikilinks
Ensure all related pages link to each other using
syntax. Every mention of an entity or concept that has its own page should be linked.
5. Update wiki/index.md
For each new page created, add an entry under the appropriate category header:
- [[Page Name]] — one-line summary (under 120 characters)
6. Update wiki/log.md
Append:
## [YYYY-MM-DD] ingest | Source Title
Processed source-filename.md. Created N new pages, updated M existing pages.
New entities: [[Entity1]], [[Entity2]]. New concepts: [[Concept1]].
7. Report results
Tell the user what was done:
- Pages created (with links)
- Pages updated (with what changed)
- New entities and concepts identified
- Any contradictions found with existing content
When processing multiple sources (batch), report aggregate results at the end:
- Total sources processed
- Total pages created and updated
- Summary of new entities and concepts across all sources
Conventions
- Tags are organic — let them emerge naturally from content. Don't force a domain taxonomy. When a source spans multiple domains, tag with all relevant domains.
- Source summary pages are factual only. Save interpretation and synthesis for concept and synthesis pages.
- A single source typically touches 10-15 wiki pages. This is normal and expected.
- When new information contradicts existing wiki content, add a
> [!warning] Contradiction
- Prefer updating existing pages over creating new ones. Only create a new page when the topic is substantive enough to warrant it.
- Use for all internal references. Never use raw file paths.
What's Next
After ingesting sources, the user can:
- Ask questions with to explore what was ingested
- Ingest more sources — clip another article and run again
- Health-check with after every 10 ingests to catch gaps
- Discover connections with to find cross-domain patterns
- Get inspired with to generate creative prompts