docs: sync documentation from private repo
- Rewrite 3_context-and-memory.md (auto-context system, remove legacy memory) - Create 7_voice.md (voice interface documentation) - Update 0_overview.md, 1_architecture.md, 4_tools-and-workflows.md, 6_ui.md, 8_mcp.md - Add simple human-readable intros to all docs - Update README.md with voice doc reference Synced from ra-h commit 4a3d7e0
This commit is contained in:
“BeeRad”
+138
-125
@@ -1,148 +1,161 @@
|
||||
# Context & Memory
|
||||
|
||||
## Context Builder
|
||||
> How RA-H decides what information to show the AI during conversations.
|
||||
|
||||
The context builder assembles the information each agent sees during conversations. It creates **cacheable blocks** (for Anthropic) and structured context for all agents.
|
||||
**How it works:** RA-H automatically identifies your 10 most-connected knowledge nodes (by edge count) and shares their titles with the AI as background context. When you focus on a specific node, the AI also sees a preview of that content. This means the AI always knows about your most important ideas without you having to manually select them.
|
||||
|
||||
### Context Structure
|
||||
---
|
||||
|
||||
Every agent receives:
|
||||
## Context System Overview
|
||||
|
||||
**1. Base Context**
|
||||
- How nodes, edges, and dimensions work
|
||||
- Node reference format: `[NODE:id:"title"]`
|
||||
- If auto-context is enabled, BACKGROUND CONTEXT explains that the 10 most-connected nodes will be listed later
|
||||
- Pronouns like "this conversation/paper/video" refer to focused node
|
||||
Every conversation includes context assembled by the **context builder**. This context tells the AI about your knowledge base, available tools, and what you're currently working on.
|
||||
|
||||
**2. Agent Instructions**
|
||||
- Agent-specific system prompt (ra-h, ra-h-easy, wise-rah, mini-rah)
|
||||
- Role-specific behavior guidelines
|
||||
- Execution approach and response style
|
||||
### What Gets Included
|
||||
|
||||
**3. Tool Definitions**
|
||||
- Role-specific tools (orchestrator/executor/planner)
|
||||
- Each tool's description and parameters
|
||||
- Usage guidelines
|
||||
| Block | Contents | Cached? |
|
||||
|-------|----------|---------|
|
||||
| **Base Context** | How nodes, edges, dimensions work; formatting rules | ✅ Yes |
|
||||
| **Agent Instructions** | Role-specific prompts (ra-h, ra-h-easy, wise-rah, mini-rah) | ✅ Yes |
|
||||
| **Tool Definitions** | Available tools and their parameters | ✅ Yes |
|
||||
| **Workflow Definitions** | Available workflows (orchestrators only) | ✅ Yes |
|
||||
| **Background Context** | Top 10 most-connected nodes (if enabled) | ✅ Yes |
|
||||
| **Focused Nodes** | Currently open node(s) with content previews | ❌ No |
|
||||
|
||||
**4. Workflow Definitions** (orchestrators only)
|
||||
- Available workflows (integrate, etc.)
|
||||
- Workflow descriptions and triggers
|
||||
- Only shown to ra-h and ra-h-easy
|
||||
---
|
||||
|
||||
**5. Background Context (auto-context hubs)** (orchestrators only)
|
||||
- Optional block controlled by `~/Library/Application Support/RA-H/config/settings.json`
|
||||
- Lists the 10 nodes with the highest edge counts (ID + title + edge count)
|
||||
- Reminds agents to call `queryNodes`/`getNodesById` for full content
|
||||
- Ordered by edge count, then most recently updated to break ties
|
||||
## Auto-Context System
|
||||
|
||||
**6. Focused Nodes**
|
||||
- Primary focused node (active tab)
|
||||
- Additional focused nodes (other open tabs)
|
||||
- 25-word content previews
|
||||
- Chunk status and embedding availability
|
||||
- Link information
|
||||
Auto-context automatically includes your most important knowledge in every conversation. It replaces the old manual "pinning" system.
|
||||
|
||||
### Context Caching
|
||||
### How It Works
|
||||
|
||||
**Anthropic (Claude):**
|
||||
- Explicit cache control blocks (`cache_control: {type: 'ephemeral'}`)
|
||||
- Caches base context, instructions, tools, workflows, background context
|
||||
- Focused nodes NOT cached (changes frequently)
|
||||
1. **Toggle:** Enable in Settings → Context tab
|
||||
2. **Query:** Finds top 10 nodes by edge count (most connections = most important)
|
||||
3. **Format:** Shows `[NODE:id:"title"] (edges: X)` for each hub node
|
||||
4. **Agent behavior:** Agents see titles only; they call `queryNodes` or `getNodesById` when they need full content
|
||||
|
||||
**OpenAI (GPT):**
|
||||
- Implicit caching based on prefix matching
|
||||
- Same block structure for consistency
|
||||
- No explicit cache control markers needed
|
||||
|
||||
### Truncation Strategy
|
||||
|
||||
- **Background context:** IDs + titles only to keep the block lightweight
|
||||
- **Focused nodes:** 25-word previews
|
||||
- **Full content access:** Agents use `queryNodes`, `getNodesById`, `searchContentEmbeddings` for complete content
|
||||
- **Chunk status indicator:** Shows if embeddings are available (avoid re-extraction)
|
||||
|
||||
## Memory System (Legacy)
|
||||
|
||||
The automatic memory pipeline has been removed. Existing memory nodes remain in the database for archival purposes, but no new ones are created and they are excluded from auto-context. The old files (`src/services/memory/**`) have been deleted along with the `ENABLE_CHAT_MEMORY_PIPELINE` toggle. Future improvements should store long-term knowledge as normal nodes with explicit dimensions rather than a background pipeline.
|
||||
6. Update checkpoint in `chat_memory_state`
|
||||
|
||||
**Location:**
|
||||
- Pipeline: `/src/services/memory/synthesis/chatMemoryPipeline.ts`
|
||||
- Trigger: `/src/services/chat/middleware.ts:168-178`
|
||||
- Extraction: `/src/services/memory/synthesis/llmSynthesis.ts`
|
||||
|
||||
### Memory Node Structure
|
||||
|
||||
```typescript
|
||||
{
|
||||
type: 'memory',
|
||||
title: 'Insight on [subject]',
|
||||
description: 'Multi-line list of facts',
|
||||
content: 'Second-person statements combined',
|
||||
metadata: {
|
||||
category: 'identity' | 'interests' | 'models' | 'preferences' | 'relationships',
|
||||
subject_type: 'person' | 'project' | 'concept' | 'resource' | 'organization' | 'workflow',
|
||||
source_thread: string,
|
||||
source_helper: string,
|
||||
importance: 'low' | 'medium' | 'high',
|
||||
focused_node_titles: string[],
|
||||
canonical_key: string | null,
|
||||
subject: string
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Memory Categories
|
||||
|
||||
- **identity** - Who you are, your roles, background
|
||||
- **interests** - What you're curious about, learning, exploring
|
||||
- **models** - How you think, mental frameworks, approaches
|
||||
- **preferences** - What you like/dislike, priorities, values
|
||||
- **relationships** - Connections to people, projects, concepts
|
||||
|
||||
### Subject Types
|
||||
|
||||
Memory extraction classifies subjects into these types:
|
||||
|
||||
- **person** - Individual people (yourself or collaborators)
|
||||
- **project** - Projects, products, or initiatives
|
||||
- **concept** - Ideas, theories, mental models, beliefs
|
||||
- **resource** - Tools, articles, books, references
|
||||
- **organization** - Companies, institutions, communities
|
||||
- **workflow** - Processes, systems, methodologies
|
||||
|
||||
### Importance Levels
|
||||
|
||||
Each memory fact is classified by importance:
|
||||
|
||||
- **high** - Explicitly marked as priority, main project, strong belief ("is crucial", "must")
|
||||
- **medium** - Normal statements and observations (default)
|
||||
- **low** - Tentative, uncertain, or exploratory statements
|
||||
|
||||
## Auto-Context Toggle
|
||||
|
||||
Auto-context replaces manual pinning. When enabled, BACKGROUND CONTEXT includes the 10 nodes with the highest edge counts.
|
||||
|
||||
**How it works:**
|
||||
1. Settings UI exposes a toggle inside the Context tab.
|
||||
2. The toggle writes to `~/Library/Application Support/RA-H/config/settings.json`.
|
||||
3. Context builder and workflows read that file on each request (missing file defaults to `false`).
|
||||
4. When enabled, the following query runs once per request:
|
||||
### The Query
|
||||
|
||||
```sql
|
||||
SELECT n.id,
|
||||
n.title,
|
||||
COUNT(DISTINCT e.id) AS edge_count,
|
||||
n.updated_at
|
||||
SELECT n.id, n.title, COUNT(DISTINCT e.id) AS edge_count
|
||||
FROM nodes n
|
||||
LEFT JOIN edges e ON (e.from_node_id = n.id OR e.to_node_id = n.id)
|
||||
WHERE n.type IS NULL OR n.type != 'memory'
|
||||
GROUP BY n.id
|
||||
ORDER BY edge_count DESC, n.updated_at DESC
|
||||
LIMIT 10;
|
||||
LIMIT 10
|
||||
```
|
||||
|
||||
**Notes:**
|
||||
- Only orchestrators (ra-h / ra-h-easy) see the block.
|
||||
- Titles + edge counts are shown; agents must call `queryNodes`/`getNodesById` for content.
|
||||
- Users who previously pinned nodes are auto-migrated to `autoContextEnabled: true` the first time the toggle helper runs and sees legacy pins.
|
||||
**Tie-breaking:** When nodes have equal edge counts, most recently updated wins.
|
||||
|
||||
### Settings Storage
|
||||
|
||||
**Location:** `~/Library/Application Support/RA-H/config/settings.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"autoContextEnabled": true,
|
||||
"lastPinnedMigration": "2025-12-09T00:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
**Legacy migration:** If you had pinned nodes before the auto-context update, the system automatically enabled auto-context on first run.
|
||||
|
||||
### Context Block Format
|
||||
|
||||
When enabled, agents see this in their system prompt:
|
||||
|
||||
```
|
||||
=== BACKGROUND CONTEXT ===
|
||||
Top 10 most-connected nodes (important knowledge hubs). Use queryNodes/getNodesById if relevant.
|
||||
|
||||
[NODE:1573:"building ra-h - knowledge management system"] (edges: 47)
|
||||
[NODE:4436:"Continual learning explains some interesting phenomena"] (edges: 32)
|
||||
[NODE:3014:"Multi-Agent Research Systems: Insights from Simon Willison"] (edges: 28)
|
||||
...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Focused Nodes
|
||||
|
||||
Focused nodes are the node(s) you currently have open in the Focus panel.
|
||||
|
||||
### What Agents See
|
||||
|
||||
- **Primary focused node:** The active tab
|
||||
- **Additional focused nodes:** Other open tabs
|
||||
- **Content preview:** First ~25 words
|
||||
- **Metadata:** Title, ID, link, dimensions, chunk status
|
||||
|
||||
### Example Format
|
||||
|
||||
```
|
||||
=== FOCUSED NODES ===
|
||||
|
||||
### Primary: [NODE:4523:"How RAG systems work"]
|
||||
Preview: Retrieval-augmented generation (RAG) combines information retrieval with language model generation to produce more accurate and grounded responses...
|
||||
Link: https://example.com/rag-systems
|
||||
Dimensions: research, ai, papers
|
||||
Chunk status: chunked (embeddings available)
|
||||
|
||||
### Also Open:
|
||||
- [NODE:4520:"Vector databases explained"] (25 words preview...)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Context Caching
|
||||
|
||||
RA-H uses provider-specific caching to reduce costs and latency.
|
||||
|
||||
### Anthropic (Claude)
|
||||
|
||||
- **Explicit cache control:** Blocks marked with `cache_control: { type: 'ephemeral' }`
|
||||
- **What's cached:** Base context, instructions, tools, workflows, background context
|
||||
- **What's NOT cached:** Focused nodes (change too frequently)
|
||||
|
||||
### OpenAI (GPT)
|
||||
|
||||
- **Implicit caching:** Based on prefix matching
|
||||
- **Same structure:** Identical blocks, just no explicit markers
|
||||
- **Optimization:** Prompts structured for maximum cache reuse
|
||||
|
||||
---
|
||||
|
||||
## Agent-Specific Context
|
||||
|
||||
Different agents receive different context based on their role:
|
||||
|
||||
| Agent | Background Context | Workflows | Tools |
|
||||
|-------|-------------------|-----------|-------|
|
||||
| **ra-h / ra-h-easy** (orchestrators) | ✅ Yes | ✅ Yes | All |
|
||||
| **wise-rah** (workflow executor) | ❌ No | ❌ No | Planner tools only |
|
||||
| **mini-rah** (worker) | ❌ No | ❌ No | Executor tools only |
|
||||
|
||||
**Why orchestrators only:** Background context helps with general conversation. Workers execute specific tasks and don't need the full picture.
|
||||
|
||||
---
|
||||
|
||||
## Key Files
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `src/services/helpers/contextBuilder.ts` | Assembles system prompts with caching |
|
||||
| `src/services/context/autoContext.ts` | Auto-context query and formatting |
|
||||
| `src/services/settings/autoContextSettings.ts` | Settings read/write helpers |
|
||||
| `src/components/settings/ContextViewer.tsx` | Settings UI for auto-context toggle |
|
||||
|
||||
---
|
||||
|
||||
## Legacy: Memory System (Removed)
|
||||
|
||||
The automatic memory extraction pipeline has been **removed**. Previously, RA-H would analyze conversations and create "memory" nodes automatically. This system was removed because:
|
||||
|
||||
1. Memory nodes cluttered the database
|
||||
2. Quality was inconsistent
|
||||
3. Users preferred explicit knowledge capture
|
||||
|
||||
**Existing memory nodes:** Still in the database but excluded from auto-context (filtered by `type != 'memory'`).
|
||||
|
||||
**Future approach:** Store long-term knowledge as regular nodes with explicit dimensions rather than automatic extraction.
|
||||
|
||||
Reference in New Issue
Block a user