docs: sync documentation from private repo

- Rewrite 3_context-and-memory.md (auto-context system, remove legacy memory)
- Create 7_voice.md (voice interface documentation)
- Update 0_overview.md, 1_architecture.md, 4_tools-and-workflows.md, 6_ui.md, 8_mcp.md
- Add simple human-readable intros to all docs
- Update README.md with voice doc reference

Synced from ra-h commit 4a3d7e0
This commit is contained in:
“BeeRad”
2026-01-05 12:42:51 +11:00
parent a3647c0693
commit 0f408bc907
8 changed files with 973 additions and 480 deletions
+125
View File
@@ -0,0 +1,125 @@
# Voice Interface
> Talk to RA-H instead of typing.
**How it works:** Press the microphone button to speak your message. RA-H converts your speech to text using OpenAI's Realtime API, sends it to the AI, and speaks the response back using text-to-speech. All processing requires an internet connection.
---
## Overview
The voice interface lets you have spoken conversations with RA-H. It uses:
- **Speech-to-Text (STT):** OpenAI Realtime API for low-latency transcription
- **Text-to-Speech (TTS):** OpenAI TTS for natural voice responses
- **Same AI agents:** Your voice messages go to the same orchestrator (Easy/Hard mode)
---
## Using Voice
### Starting a Voice Session
1. Click the **microphone icon** in the chat panel
2. Grant microphone permission if prompted
3. Speak naturally — RA-H transcribes in real-time
4. The AI responds with both text and audio
### Visual Feedback
- **"RA-H is listening"** strip appears when active
- **Amplitude bars** show microphone input level
- **Transcript preview** shows what's being recognized
### Stopping Voice Input
- Click the microphone button again
- Or wait for silence detection (~800ms pause)
---
## Requirements
| Requirement | Details |
|-------------|---------|
| **API Key** | OpenAI API key (same key used for Easy mode) |
| **Internet** | Required for STT and TTS |
| **Microphone** | Mac app requests permission on first use |
| **macOS** | 12+ (Monterey or later) |
---
## Cost
Voice features use OpenAI's APIs which have usage costs:
| Feature | Pricing |
|---------|---------|
| **Realtime STT** | Included in Realtime API usage |
| **TTS** | ~$0.015 per 1,000 characters |
Costs are tracked in:
- Per-message metadata (`voice_tts_*` fields)
- `voice_usage` SQLite table
- Settings → Analytics panel
---
## Limitations
- **Internet required** — No offline voice support
- **English optimized** — Other languages may have lower accuracy
- **No voice selection** — Uses default OpenAI voice
- **Mac only** — Voice features not available in web/open-source version
---
## Technical Details
### API Endpoints
| Endpoint | Purpose |
|----------|---------|
| `/api/realtime/ephemeral-token` | Get temporary token for Realtime API |
| `/api/voice/tts` | Convert text to speech |
### Key Files
| File | Purpose |
|------|---------|
| `src/components/agents/hooks/useRealtimeVoiceClient.ts` | STT WebSocket client |
| `src/components/agents/hooks/useAssistantTTS.ts` | TTS playback |
| `app/api/realtime/ephemeral-token/route.ts` | Token endpoint |
| `app/api/voice/tts/route.ts` | TTS endpoint |
### Environment Variables
```bash
# Required for voice
OPENAI_API_KEY=sk-...
# Optional: cost tracking
RAH_TTS_COST_PER_1K_CHAR_USD=0.015
```
---
## Troubleshooting
### "Microphone not working"
1. Check System Preferences → Privacy → Microphone → RA-H is allowed
2. Restart the app after granting permission
3. Test microphone in other apps
### "Voice isn't responding"
1. Check internet connection
2. Verify OpenAI API key is valid
3. Check Settings → API Keys
### "Transcription is inaccurate"
- Speak clearly and at normal pace
- Reduce background noise
- Voice works best in quiet environments