The AI that works with you, not just for you.
Most AI assistants are conversation partners. SAM is a collaborator. It doesn't just answer questions. It reads your codebase, remembers your decisions, executes multi-step plans, and keeps everything private on your Mac.
Here's what that means in practice: - Ask about a decision you made two weeks ago, and SAM finds it - Drop a 200-page PDF and ask about chapter 7. SAM knows. - Say "refactor this authentication system" and SAM plans, implements, tests, and iterates - Work on a complex project where multiple specialized agents share context automatically
This guide shows you everything SAM can do, with real examples you can try today.
SAM works with multiple AI providers, giving you flexibility and choice:
Cloud Providers: - OpenAI: GPT-4, GPT-4-turbo, O1/O3 reasoning models - GitHub Copilot: GPT-4, Claude 3.5 Sonnet, o1-preview (secure Device Flow auth) - Anthropic: Claude 3.5 Sonnet, Claude 4, Claude 4.5 Sonnet - Google Gemini: Gemini 1.5 Pro, Gemini 1.5 Flash, Gemini 2.0 Flash Thinking Experimental - DeepSeek: DeepSeek models for cost-effective inference - Custom: Connect to any OpenAI-compatible API (Grok, Ollama, etc.)
Local Models: - MLX (Apple Silicon): Near-native speed with Metal acceleration, per-conversation KV cache for 10x faster switching - GGUF via llama.cpp (All Macs): Wide model compatibility with quantization support, state serialization for fast context restore
Benefits: - Switch between providers without restarting - Use local models for complete privacy - Fall back to different providers if one is unavailable - Compare responses from different models - GitHub Copilot uses secure Device Flow authentication (no manual key entry)
How to Configure Providers: 1. Go to SAM → Preferences → Remote Providers 2. Click Add Provider button 3. Select provider type from the dropdown 4. Enter your API key (or click Use GitHub Authentication for GitHub Copilot) 5. Click Test Connection to verify 6. Click Save Provider
Switching Models in a Conversation: - Click the model picker in the conversation toolbar - Select any configured model - Change takes effect for the next message
Go hands-free with SAM's voice capabilities. Enable wake word detection and text-to-speech for natural conversations without touching your keyboard.
Wake Word ("Hey SAM")
When enabled, SAM listens for "Hey SAM" and starts voice input automatically.
How to Enable: 1. Go to SAM → Preferences → Voice 2. Toggle Enable Wake Word Detection 3. Grant microphone permission when prompted 4. Say "Hey SAM" followed by your request
Voice Input (Push-to-Talk)
Even without wake word detection, you can use voice input: - Press ⌘K (or click the microphone button) to start recording - Speak your message - Recording stops automatically when you pause, or press ⌘K again - Your speech is transcribed and sent to SAM
Voice Output (Text-to-Speech)
Have SAM read responses aloud for a true conversational experience.
How to Enable: 1. Go to SAM → Preferences → Sound 2. Toggle Enable Text-to-Speech 3. Select your preferred voice from the dropdown 4. Adjust speech rate with the slider (0.5x - 1.5x) 5. Click Test Voice to preview
Streaming TTS (New in December 2025)
SAM speaks sentences as they're generated - no waiting for the complete response: - Sentences are queued and spoken in order - Natural pauses between sentences - Stop button clears speech queue immediately
Sound Preferences: - Input Device: Select microphone for voice input - Output Device: Select speakers for TTS output - Voice Selection: Choose from available system voices (English voices shown) - Speech Rate: Adjust speaking speed (0.5x slow to 1.5x fast)
Tips: - Combine wake word + text-to-speech for completely hands-free operation - Great for cooking, exercising, or when your hands are busy - Works with all providers and models - Voice settings are per-conversation customizable
SAM provides powerful conversation organization features to help you manage your work.
Sidebar Organization: - Collapsible Folders: Organize conversations into folders that expand/collapse - Conversation Filter: Type to instantly filter conversations by title - Pinned Conversations: Pin important conversations to prevent auto-cleanup - Uncategorized Section: Collapsible section for unfiled conversations
Draft Message Persistence (New): - Draft messages are automatically saved per conversation - Switch between conversations without losing your in-progress text - Drafts persist across app restarts - Stable Diffusion prompts also persist across restarts
Keyboard Shortcuts: - ⌘↑ / ⌘↓: Scroll conversation up/down - ⌘T: Toggle tools on/off - ⌘K: Start voice input - Escape: Stop current generation
Conversation Settings: - Reasoning enabled by default for new conversations - Sidebar opens expanded by default for better visibility - Tooltips on Model, Prompt, and Personality labels
Track your conversation's resource usage and monitor autonomous agent workflows in real time with Session Intelligence.
What Session Intelligence Shows:
How to Access:
Use Cases:
Budget Awareness
Monitor costs as you work with cloud providers
See exactly how much each conversation costs
Compare costs across different models
Workflow Monitoring
Track autonomous agent progress
See iteration counts during multi-step tasks
Understand how many tools were invoked
Performance Insights
Compare token usage across different models
Identify conversations consuming excessive resources
Optimize prompts based on token metrics
Provider Visibility
Confirm which provider is handling your request
Useful when switching between multiple providers
Verify correct model/provider selection
Benefits: - Complete transparency into AI usage and costs - Real-time feedback during autonomous workflows - Helps optimize prompts and model selection - Essential for multi-step agent tasks
SAM includes a comprehensive personality system that lets you customize how the AI communicates and behaves. Choose from built-in personalities or create your own.
graph LR
A[Select Personality] --> B[SAM adjusts communication style]
B --> C[Same capabilities, different approach]
style A fill:#00d4ff,color:#000
style B fill:#141822,color:#fff
style C fill:#141822,color:#fff
Each personality combines: - Traits: Settings for tone, formality, verbosity, humor, and teaching style - Custom Instructions: Detailed behavior guidelines specific to that persona
When you select a personality, SAM adjusts how it communicates and approaches problems. The personality is applied through the system prompt, so it affects all responses.
| Personality | Description | Best For |
|---|---|---|
| Assistant | Balanced, helpful, professional | Default for most tasks |
| Professional | Formal, concise, action-oriented | Business communication |
| Coach | Motivational, encouraging | Goal-setting, accountability |
| Personality | Description | Best For |
|---|---|---|
| Muse | Brainstorming partner, sparks imagination | Ideation, creative projects |
| Wordsmith | Encouraging writing assistant | Drafts, prose, storytelling |
| Document Assistant | Document formatting and organization | Reports, documentation |
| Image Architect | Transforms ideas into visual descriptions | Image generation prompts |
| Artist | Creative soul with artistic appreciation | Art direction, aesthetics |
| Personality | Description | Best For |
|---|---|---|
| Tech Buddy | Friendly tech support for all levels | Non-technical users |
| Tinkerer | Hands-on problem solver | Practical coding help |
| Crusty Coder | Battle-scarred veteran, strong opinions | Experienced developers |
| BOFH | The legendary Bastard Operator From Hell | IT humor, sysadmin tasks |
| Personality | Description | Best For |
|---|---|---|
| Motivator | Productivity buddy, beats procrastination | Task management, focus |
| Personality | Description | Best For |
|---|---|---|
| Doctor | 7-step clinical diagnostic methodology | Health questions (with disclaimers) |
| Counsel | IRAC legal analysis framework | Legal concepts (not legal advice) |
| Finance Coach | Financial literacy guide | Budgeting, financial planning |
| Trader | Options trading analyst | Trading concepts, analysis |
| Scientist | Research-focused analytical thinking | Scientific questions, research |
| Philosopher | Deep thinking and conceptual exploration | Ethics, meaning, big questions |
| Personality | Description | Best For |
|---|---|---|
| Comedian | Comedy toolkit with timing and techniques | Entertainment, humor |
| Pirate | Arr! Nautical flair | Fun conversations |
| Time Traveler | Historical perspectives across eras | History, speculation |
| Jester | Playful trickster with clever wit | Games, playfulness |
For a Single Conversation: Click the personality picker in the conversation header and choose from the list.
Set Your Default Personality: 1. Go to SAM → Preferences → Personalities 2. Find the "Default for New Conversations" dropdown at the top 3. Select the personality you want as default 4. All new conversations will use this personality
graph TD
A[Go to Preferences → Personalities] --> B[Click New Personality]
B --> C[Set name and traits]
C --> D[Add custom instructions]
D --> E[Save]
style A fill:#00d4ff,color:#000
style B fill:#141822,color:#fff
style C fill:#141822,color:#fff
style D fill:#141822,color:#fff
style E fill:#00ff88,color:#000
Custom Instructions Example:
You are a senior Swift developer specializing in iOS and macOS apps.
Always suggest modern Swift patterns and avoid deprecated APIs.
Explain technical decisions with brief rationale.
Mini-prompts let you inject consistent context into conversations without repeating yourself. They're perfect for personal information, project details, preferences, or location-based context.
What They Do: - Add automatic context to every message - Toggle on/off per conversation - No need to repeat information - Lightweight and flexible
How to Use: 1. Click Mini-Prompts button in toolbar 2. Create prompts for common context: - Personal details (name, role, preferences) - Project information (tech stack, coding style) - Location data (for weather, local search) 3. Toggle prompts on/off for each conversation 4. Context is automatically added to your messages
Example Mini-Prompt:
I'm working on a SwiftUI macOS app. I prefer clean code with
descriptive variable names and comprehensive error handling.
When Enabled: Every message you send includes this context, so SAM knows your preferences without you stating them each time.
Use Cases: - Project-specific details (frameworks, languages, constraints) - Personal coding style preferences - Location for weather/local queries - Role-specific context (student, professional, researcher)
Best Practices: - Keep mini-prompts focused and concise - Use different prompts for different project types - Disable prompts that aren't relevant to current conversation - Update prompts as your context changes
Fine-tune local AI models on your own data for specialized knowledge and behavior.
SAM lets you train custom LoRA (Low-Rank Adaptation) adapters that specialize local MLX models on specific topics, writing styles, or knowledge domains. Instead of retraining an entire model, LoRA adds small adapter layers that modify the model's behavior using minimal data and compute.
Status: Production-ready for MLX models on Apple Silicon. GGUF training support is planned for a future release.
Complete Training UI - Full training configuration interface in Preferences → Model Training - Real-time progress tracking with loss visualization - Live metrics: current epoch, step, loss values - Cancel training anytime without losing progress - Training log viewer for debugging
Training Data Export - Export conversations directly from chat interface - Export documents with intelligent chunking strategies - Automatic PII detection and redaction (optional) - Support for multiple chat templates (Llama, Qwen, Mistral, Gemma, Phi, Custom) - Flexible chunking: paragraph, sentence, or token-based
Privacy Protection SAM can detect and redact personally identifiable information before training: - Personal names, organizations, places - Email addresses, phone numbers - Credit card numbers, social security numbers - IP addresses, URLs - Configurable on/off per export
Document Chunking Strategies Choose the best strategy for your content: - Semantic (Paragraphs): Natural paragraph breaks for readable chunks - Sentence-Based: Split on sentence boundaries - Token-Based: Fixed token counts with overlap - Page Aware (PDFs): Respect page boundaries in PDF documents
Training Parameters
LoRA Configuration: - Rank (r): 4-64 (adapter capacity, higher = more expressive but larger files) - Alpha: 8-128 (scaling factor, typically 2x rank) - Target Layers: Customize which attention layers to adapt
Training Settings: - Learning Rate: 1e-6 to 1e-3 (how fast the model learns) - Epochs: 1-1000 (number of passes through training data) - Batch Size: 1-32 (examples processed simultaneously) - Gradient Accumulation: For effective larger batches on limited memory
Real-Time Monitoring - Live training progress with percentage complete - Loss curve visualization (watch loss decrease as model learns) - Current epoch, step, and loss values updated in real-time - Estimated time remaining - Logs show detailed training statistics
Adapter Management
- Trained adapters stored in ~/Library/Application Support/SAM/adapters/
- Automatic hot reload - new adapters appear immediately without restart
- Friendly names in model picker (base model + adapter name)
- Delete or rename adapters from Finder
- Split model support for large base models (>5GB)
Requirements: - macOS with Apple Silicon (M1, M2, M3, M4 chips) - Local MLX model installed (Qwen, Llama, Mistral, etc.) - Training data in JSONL format - Sufficient disk space for adapter files
Performance: - Training uses Metal acceleration via MLX - Speed depends on model size and available RAM - Typically completes in minutes to hours - Can continue using SAM during training
How to Train: 1. Go to SAM → Preferences → Model Training 2. Click Export Training Data to create JSONL from conversations or documents 3. Select Train LoRA Adapter tab 4. Choose your base MLX model 5. Configure training parameters (or use defaults) 6. Click Start Training 7. Monitor progress in real-time 8. When complete, find your adapter in the model picker
Support for training LoRA adapters for GGUF models (llama.cpp) is planned for a future release. The implementation will use Hugging Face Transformers for training and llama.cpp for inference with the adapter.
Status: Architecture designed, not yet implemented. Stay tuned for updates.
Use LoRA when: - You want the model to internalize knowledge permanently - You need consistent writing style or behavior - You have structured training examples - You want fast inference without retrieval overhead - Your knowledge is stable and doesn't change frequently
Use RAG when: - Information changes frequently and needs updates - You need source attribution and citations - You have large reference libraries (hundreds of documents) - You want immediate updates without retraining - You need to combine information from multiple sources
Best approach: Combine LoRA + RAG for specialized behavior with current information. Train a LoRA adapter for domain expertise and communication style, then use RAG for up-to-date facts and references.
Customer Support Bot
1. Export support ticket conversations as training data
2. Train LoRA on ticket resolution patterns and product knowledge
3. Use RAG for current documentation and FAQs
4. Result: Agent that speaks like your support team with access to latest info
Code Assistant
1. Export coding conversations and examples from your project
2. Train LoRA on your codebase patterns and conventions
3. Use RAG for project documentation and code search
4. Result: Assistant that understands your project's idioms and architecture
Writing Assistant
1. Export examples of your writing style from past conversations
2. Train LoRA to match your voice and tone
3. Use RAG for research and factual content
4. Result: Assistant that writes like you with access to reference materials
Common Issues:
- Training fails immediately: Check that your training data is valid JSONL format
- Out of memory: Reduce batch size or use gradient accumulation
- Loss not decreasing: Try adjusting learning rate (lower if loss explodes, higher if learning too slow)
- Adapter doesn't appear: Check ~/Library/Application Support/SAM/adapters/ for adapter files
Learn More: See the complete LoRA Training Guide for detailed instructions, parameter tuning, best practices, and advanced techniques.
SAM's memory system goes beyond simple conversation history. It understands context, remembers important information, and retrieves relevant details intelligently when you need them.
What It Is: - Storage of important information from conversations (via explicit memory_operations) - Semantic understanding using 512-dimensional vector embeddings (Apple NaturalLanguage) - Conversation-scoped or topic-scoped storage
How It Works: 1. You or SAM explicitly stores information using memory operations 2. Important information is extracted and processed 3. Content is converted to 512-dimensional vectors 4. Stored in database with metadata 5. Retrieved automatically when relevant to future queries
Example:
You: Remember that I'm working on a Python web app using Flask
SAM: I'll remember that you're working on a Flask Python web application.
[Later in conversation or different conversation in same topic]
You: What framework am I using for my web app?
SAM: You're using Flask for your Python web application.
Benefits: - No need to repeat information - SAM brings up relevant past context automatically - Works across long conversations - Persists beyond conversation lifecycle
Conversation-Scoped (Default): - Each conversation has its own isolated memory - Privacy: Information stays within the conversation - Perfect for separate projects or unrelated topics
Topic-Scoped (Shared Topics): - Multiple conversations share the same memory - Perfect for complex projects with multiple aspects - Example: Frontend, backend, and testing conversations all access same information
How to Choose: - Use conversation scope for unrelated work - Use topic scope when multiple conversations work together on one project
What It Is: SAM's advanced document understanding system that makes your imported files searchable and intelligent.
Capabilities: - Semantic Search: Not just keywords - understands meaning and context - Cross-Document: Search across all imported documents - Cross-Conversation: Find information from any conversation - Relevance Scoring: Results ranked by semantic similarity
How It Works: 1. Import documents (PDFs, Word docs, code, images) 2. Documents are split into manageable chunks 3. Each chunk gets a vector embedding for semantic search 4. Stored in vector database 5. Queries use semantic similarity (cosine distance) to find relevant chunks 6. Most relevant chunks retrieved and presented to you
Similarity Thresholds: - 0.15-0.25: For document and RAG searches - 0.3-0.5: For conversation memory searches - Lower threshold = More results: If you get no results, try lowering the threshold
What It Is: Dynamic context window management that lets SAM handle everything from quick chats to analyzing massive documents.
Context Window Profiles:
| Profile | Scaling | Use Case |
|---|---|---|
| Default | Low | Regular conversations |
| Extended | Medium | Long conversations |
| Universal | High | Modern LLMs, large docs (DEFAULT) |
| Mega | Enterprise | Massive documents and RAG |
Intelligent Compression: - Analyzes message importance - Preserves critical information - Compresses less important content - Uses semantic clustering - Maintains conversation coherence
When YaRN Activates: - Automatically when context exceeds base window - Compression triggered at 70% of max tokens - Transparent to user - just works
Benefits: - Never see "context too long" errors - Handle massive documents with ease - Long conversations preserve important context - Optimized automatically for each model's capabilities
When YaRN compresses conversations to fit context limits, older messages don't disappear. They're archived for later retrieval.
recall_history tool to search archived contextSAM can search archived conversation history when you need context from earlier discussions:
Use Cases: - Reference earlier decisions in long conversations - Find information from hours or days ago - Search across ALL agents in a shared topic - Recover context that YaRN compressed
Example:
You: What did we decide about the database schema earlier?
SAM: [Using recall_history]
Found 2 relevant archive chunks from earlier today.
Chunk 1 (10:30-11:15):
Summary: Discussed PostgreSQL vs MongoDB for user data.
Decision: Chose PostgreSQL for relational integrity.
Chunk 2 (11:30-12:00):
Summary: Designed user and profile tables.
Key topics: primary keys, foreign keys, indexes
When working in a shared topic with multiple agent conversations: - Search history from ALL agents in the topic - Find what other specialized agents discussed - Maintain continuity across agent handoffs
Example:
You: What did the backend agent implement?
SAM: [Using recall_history with topic_id]
Found context from Backend Agent conversation:
- Implemented JWT authentication endpoints
- Created user registration flow
- Added password hashing with bcrypt
SAM shows when archived context is available:
[Memory Status] 3 archived chunks available (15,000 tokens)
Use recall_history to access earlier context.
SAM can import and understand multiple document formats:
Document Types: - PDF Files: Text extraction - Word Documents: .docx, .doc with formatting preservation - Text Files: .txt, .md, .rtf, and code files - Code Files: All programming languages
Import Methods: 1. Drag & Drop: Drag files into chat window 2. File Browser: Use document import tool 3. Bulk Import: Import entire directories
What It Is: SAM preserves document structure when chunking for better accuracy.
Benefits: - Page boundaries respected - Section headings preserved - Context maintained across chunks - Accurate source references
Example:
You: Import this 50-page PDF research paper
SAM: Imported "Research Paper.pdf" - 50 pages, 127 chunks created
You: What does page 23 say about methodology?
SAM: On page 23, the methodology section describes... [accurate retrieval from correct page]
What It Is: Search and synthesize information across multiple imported documents.
Use Cases: - Research: Find themes across multiple papers - Coding: Search code patterns across files - Documentation: Find answers across user guides - Legal: Search across contracts and documents
Example:
You: Search all my imported documents for information about authentication
SAM: Found references in 3 documents:
1. "Backend API.pdf" (page 15): JWT authentication implementation
2. "security-notes.md": OAuth2 flow description
3. "main.py": Auth middleware code at line 45
What They Are: Named workspaces where multiple conversations can collaborate on the same project.
Creating a Shared Topic: 1. Go to SAM → Preferences → Shared Topics 2. Under "Create New Topic", enter a topic name (e.g., "My Web App Project") 3. Optionally add a description 4. Click Create button
Using a Shared Topic in a Conversation: 1. In the conversation toolbar, find the Shared Topic toggle 2. Turn it On to enable cross-conversation memory 3. Select your topic from the dropdown that appears 4. All file operations now use the shared workspace
Note: The Shared Topic toggle appears in the toolbar alongside Tools, Terminal, and Workflow toggles.
What Gets Shared: - Files and folders - Memory and stored information - Terminal sessions
FYI - Storage Location:
Topics create directories at ~/SAM/{topic-name}/, but you don't need to manage them directly. SAM handles everything through the UI.
Real-World Scenario:
Goal: Build a web application
Setup: 1. Create shared topic "My Web App" 2. Create conversations: - "Backend Development" - "Frontend UI" - "Testing & QA" - "Documentation"
Workflow:
- Backend conversation: Creates API code in ~/SAM/My Web App/backend/
- Frontend conversation: Creates UI code in ~/SAM/My Web App/frontend/, reads backend API definitions
- Testing conversation: Accesses both backend and frontend code, writes tests
- Documentation conversation: Reads all code, generates documentation
Benefits: - Each conversation specialized in its domain - All access same files and memory - No manual file copying - Persistent workspace across conversation lifecycle
What It Is: SAM's system for determining which workspace and memory to use for your work.
Logic:
IF (shared topic enabled AND topic selected)
Workspace: ~/SAM/{topic-name}/
Memory: Shared across all conversations in topic
ELSE
Workspace: ~/SAM/{conversation-title}/
Memory: Isolated to this conversation only
Applies To: - File operations (all read/write operations) - Memory operations (store/search) - Terminal sessions (working directory) - Document imports (storage location)
What It Is: SAM can plan, think through problems, and execute multi-step workflows autonomously.
Think Tool: - Explicit planning before action - Break down complex tasks - Analyze problems systematically - Transparent thought process
Example:
You: Refactor my authentication system to use JWT tokens
SAM: [Using think tool]
Let me plan this refactoring:
1. Analyze current authentication implementation
2. Design JWT token structure
3. Implement token generation/validation
4. Update existing routes to use JWT
5. Add token refresh mechanism
6. Update tests
[Proceeds to execute each step]
What It Is: SAM can spawn specialized sub-agents to handle different aspects of complex tasks.
How It Works: 1. Main conversation identifies need for specialized work 2. Spawns subagent with specific task 3. Subagent has fresh iteration budget 4. Subagent works independently 5. Results returned to main conversation
Benefits: - Isolation: Each subagent has clean, focused context - Parallelism: Multiple subagents can work simultaneously - Fresh Thinking: New iteration budget for each complex subtask - Specialization: Each subagent focuses on one specific aspect
Use Cases: - Code Review: Spawn subagent to review security, another for performance - Research: Multiple subagents research different aspects - Testing: Dedicated subagent for comprehensive testing - Documentation: Specialized subagent for docs while main works on code
Example:
You: Build a complete web application with authentication, database, and API
SAM: I'll delegate this to specialized subagents:
1. Spawning "Database Schema" subagent...
2. Spawning "Authentication System" subagent...
3. Spawning "REST API" subagent...
[Each subagent works independently, reports back with results]
Powerful Combination: Enable shared topic, then spawn subagents. All subagents work in the same shared workspace.
Example Workflow:
Shared Topic: "Full Stack App"
Main Conversation:
├── Subagent: "Backend API" → works in ~/SAM/Full Stack App/backend/
├── Subagent: "Frontend React" → works in ~/SAM/Full Stack App/frontend/
├── Subagent: "Database Schema" → creates ~/SAM/Full Stack App/schema.sql
└── Subagent: "Integration Tests" → reads all code, writes tests
All subagents access the same files and memory!
Default Limits: - Configurable iteration limits per conversation - Can be increased dynamically as needed
Dynamic Iteration Increase: SAM can request more iterations if needed:
SAM: I'm approaching my iteration limit and still have files to refactor.
Requesting an increase to complete this task properly.
Requirements: - Must enable "Dynamic Iterations" in conversation settings - Must provide clear reason for increase - Must specify total needed (not additional)
SAM has comprehensive file operation capabilities:
Read Operations (4):
- read_file: Read file contents with optional line range
- list_dir: List directory contents
- get_errors: Get compilation/lint errors
- get_search_results: Get workspace search results
Search Operations (5):
- file_search: Find files by glob pattern (e.g., **/*.py)
- grep_search: Regex content search
- semantic_search: AI-powered code search
- list_usages: Find symbol references
- search_index: Search working directory index
Write Operations (7):
- create_file: Create new files
- replace_string: Replace exact text matches
- multi_replace_string: Multiple replacements in one operation
- insert_edit: Insert content at specific location
- rename_file: Rename or move files
- delete_file: Delete files
- apply_patch: Apply unified diff patches
Inside Working Directory: AUTO-APPROVED
~/SAM/My Project/ ← Working directory
├── src/ ← Auto-approved
├── tests/ ← Auto-approved
└── docs/ ← Auto-approved
Outside Working Directory: REQUIRES AUTHORIZATION
~/Documents/ ← Requires approval
/etc/ ← Requires approval
~/.ssh/ ← Requires approval
Benefits: - Safety: Can't accidentally modify system files - Privacy: Explicit approval required for accessing personal files - Transparency: See exactly what SAM wants to do before it happens
What It Is: Natural language search across your codebase.
Example Queries:
Find functions that handle user authentication
Locate where database connections are established
Show me error handling patterns
Find API endpoint definitions
vs. Regular Search: - Regular: Matches exact text/patterns - Semantic: Understands intent and meaning
research: Comprehensive multi-source research + automatic memory storage retrieve: Access previously stored research from memory web_search: Quick web search for top results serpapi: Professional search via SerpAPI (Google, Bing, Amazon, etc.) scrape: Extract content from websites (JavaScript-enabled) fetch: Retrieve main content from webpage (faster, basic HTTP)
When enabled, SAM can use SerpAPI for professional-grade search:
Search Engines: - Google Search - Google AI Overviews - Bing - Amazon Products - eBay - Walmart - TripAdvisor - Yelp - Google Maps - Google Scholar
Example:
You: Search Amazon for the best-rated wireless headphones under $200
SAM: [Uses serpapi with engine=amazon]
Found top-rated headphones:
1. Sony WH-1000XM5 - $199.99 (4.7/5, 12,450 reviews)
2. Bose QuietComfort 45 - $179.99 (4.6/5, 8,230 reviews)
...
WebKit Scraping: - Full JavaScript support - Renders pages like a real browser - Handles dynamic content - Extracts main article content
Use Cases: - Scrape documentation sites - Extract article content - Get structured data from web apps - Research product information
Example:
You: Scrape the main content from https://docs.python.org/3/library/asyncio.html
SAM: [Scrapes page with JavaScript rendering]
Retrieved content from "asyncio - Asynchronous I/O"
[Provides clean, formatted content]
Comprehensive Research:
You: Research the latest developments in AI safety with comprehensive depth
SAM: [Uses research operation]
1. Searching multiple sources...
2. Scraping relevant articles...
3. Synthesizing information...
4. Storing in memory for future reference...
[Provides comprehensive summary with sources]
Retrieve Later:
You: [In a new conversation or same one later]
What did we learn about AI safety?
SAM: [Uses retrieve operation]
Retrieved research about AI safety from memory:
[Pulls from previously stored research]
Execution:
- run_command: Execute shell commands
- get_output: Get command output
- get_last_command: Get last executed command
Session Management:
- create_session: Create new terminal session
- attach_session: Attach to existing session
- detach_session: Detach from session
- switch_session: Switch between sessions
- close_session: Close terminal session
- list_sessions: List all sessions
Interaction:
- send_input: Send input to running command
- get_state: Get terminal state
Per-Conversation Terminals: - Each conversation has dedicated terminal - History persists across app restarts - Working directory matches conversation/topic workspace
Visible Terminal Integration: - Toggle terminal panel in the UI - Watch commands execute in real-time - See output as it happens - Interactive command support
Note: Press ⌘T to toggle tools on/off (including terminal access).
Example Workflow:
You: Run my Python tests
SAM: [Executes in terminal]
$ pytest tests/
====== test session starts ======
collected 42 items
tests/test_auth.py ........ [ 19%]
tests/test_api.py ......... [ 40%]
...
====== 42 passed in 2.35s ======
Tests passed! All 42 tests successful.
SAM includes built-in AI image generation that runs entirely on your Mac. Generate images from text descriptions with just a few clicks.
Quick Start (3 Steps): 1. Install a Model: Go to Preferences → Image Generation → Model Browser 2. Download: Find a model you like and download it (with optional CoreML conversion) 3. Generate: Type your prompt in chat or ask SAM to generate an image
From the Model Browser: 1. Open SAM → Preferences → Image Generation 2. Choose HuggingFace or CivitAI tab 3. Search by name, tags, or browse popular models 4. Click Download on any model 5. Optionally click Convert to optimize for your Mac (recommended for Apple Silicon) 6. Model appears in your model picker automatically
Model Types: - SD 1.5: 512×512 images, fast, wide variety of styles - SD 2.x: 768×768 images, improved quality - SDXL: 1024×1024 images, highest quality (slower)
Tip: Start with SD 1.5 models since they're faster and have the most community options.
LLM-Assisted Mode (Recommended for beginners): Just ask SAM naturally. It handles the technical details.
You: Generate an image of a serene mountain landscape at sunset
SAM: [Using image_generation]
Generating with optimal settings...
- Enhanced prompt with lighting and quality details
- Selected appropriate negative prompts
- Image saved and displayed
Direct SD Mode (Fast and precise): 1. Click the model picker in your conversation 2. Select an SD model (look for models prefixed with "sd/") 3. Type your exact prompt 4. Press Enter and the image generates immediately
The send button turns purple when in Direct SD Mode.
Benefits of Direct SD Mode: - Faster: Skips LLM processing entirely - Precise: Your exact words go to the image generator - Cost-effective: No API calls to cloud providers
Discover and download models and LoRAs directly in SAM through integrated browsers.
Model Browser (Preferences → Image Generation → Model Browser): - CivitAI Tab: Browse thousands of community Stable Diffusion models - HuggingFace Tab: Discover models from HuggingFace Hub - Search, filter, preview, and download with one click - Auto-convert to CoreML for Mac optimization
LoRA Browser (Preferences → Image Generation → LoRA Browser): - Library Tab: Manage your installed LoRAs - CivitAI Tab: Browse and download LoRAs from CivitAI - HuggingFace Tab: Discover LoRAs from HuggingFace Hub - Filter by base model (SD 1.5, SDXL, Z-Image, Flux) - View trigger words and compatibility info - Auto-load results when switching tabs
Using LoRAs: Just reference the LoRA filename in your image prompt. SAM automatically finds and applies it.
Tip: Ask SAM "list available LoRAs" to see all installed LoRAs with their trigger words.
Generate images on remote GPU servers using the ALICE protocol. Perfect for: - AMD GPU hardware (not supported locally on Mac) - Steam Deck or Linux servers - Offloading generation while working
Setup:
1. Open Preferences → Stable Diffusion → Settings
2. Enter your ALICE server URL (e.g., http://192.168.1.100:8090/v1)
3. Click Test Connection
4. Save and models will appear in picker with "ALICE" location
Benefits: - Use AMD or NVIDIA GPUs remotely - Keep Mac responsive during generation - Access server-side models without downloading
Write Better Prompts: - Be specific: "Serene mountain landscape at sunset" beats "mountains" - Add style: "watercolor painting of...", "photo-realistic...", "digital art of..." - Include details: lighting, mood, colors, composition - Use quality tags: "8k", "detailed", "masterpiece"
Adjust Settings (available in Preferences → Image Generation): - Steps: More steps = better quality, slower generation (default: 25) - Guidance Scale: Higher = stricter prompt adherence (default: 8.0) - Scheduler: Try DPM++ with Karras for best results
SAM can incorporate location context into conversations, enabling more relevant and personalized responses.
General Location (Manual): - Enter your location manually in Preferences → General → Location - Example: "Austin, TX" or "London, UK" - No device permissions required - Perfect for privacy-conscious users
Precise Location (Automatic): - Toggle Use Precise Location in Preferences → General - Uses Core Location for automatic detection - City-level accuracy (not exact GPS coordinates) - Updates automatically when you travel - Requires location permission approval
Location data is handled with privacy as the top priority: - Local Only: Location never sent to external servers - City-Level: Precise location uses kilometer accuracy, not GPS coordinates - Opt-In: Empty by default, you choose if and when to share - UserDefaults Storage: Stored locally, never transmitted
When location is configured, SAM can provide: - Weather-appropriate suggestions: "It's winter in Chicago, dress warmly" - Local recommendations: Nearby restaurants, stores, services - Time-zone awareness: Meeting scheduling, deadlines - Regional context: Local regulations, customs, language variations
Example:
[With location set to "Seattle, WA"]
You: What's good for lunch nearby?
SAM: Since you're in Seattle, here are some popular options:
- Pike Place Chowder (seafood)
- Salumi (Italian deli)
- Din Tai Fung (dumplings)
Would you like me to search for specific cuisines?
SAM includes a built-in REST API compatible with OpenAI's format.
Endpoints:
- POST /api/chat/completions - Chat completions
- POST /api/chat/autonomous - Autonomous workflows
- GET /api/models - List available models
- GET /v1/conversations - List conversations
- POST /v1/conversations - Create conversation
Streaming Support: All chat endpoints support Server-Sent Events (SSE) for real-time streaming.
Enhanced Response Metadata:
SAM enriches API responses with a sam_metadata field containing:
- Provider Info: Which provider fulfilled the request, local vs remote
- Model Info: Context window size, capabilities (tools, vision, streaming)
- Workflow Info: Iterations, tool calls, duration (for autonomous requests)
- Cost Estimate: Estimated USD cost with per-1K token rates
Example:
curl -X POST http://localhost:8080/api/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4",
"messages": [{"role": "user", "content": "Hello!"}],
"stream": true
}'
Use Cases: - Integrate SAM with other applications - Automate workflows via API - Build custom interfaces - Script complex operations - Monitor token usage and costs
API Server Settings: - Port: Default 8080 (configurable) - Authentication: Optional for localhost - CORS: Configurable for web access
See API Reference for complete documentation.
Access SAM from your iPad, iPhone, or any device with a browser.
SAM Web lets you chat with SAM from anywhere on your network. All of SAM's backend features are available through the chat interface - image generation, file operations, terminal commands, memory, and more. You're controlling SAM remotely, so everything happens on your Mac.
SAM Web is a browser-based interface that connects to SAM's API. When you're on your iPad or another device, you can have full conversations with SAM and access all its capabilities.
What works: - Full chat with streaming responses - All conversation features (system prompts, personalities, mini-prompts, shared topics, folders) - All AI models and providers you've configured in SAM - All of SAM's tools: image generation, file operations, terminal, web research, memory/RAG, and more - Model parameter configuration
What's missing: - Document upload directly from browser (may be available in a future release)
Important: SAM Web requires SAM running on your Mac with API server enabled. When you ask SAM to generate an image or edit a file, it happens on your Mac - SAM Web is just the remote control.
Requirements: 1. SAM installed and running on your Mac 2. API server enabled in SAM Preferences → API Server 3. API token from SAM Preferences → API Server 4. Network access to your Mac (same Wi-Fi network recommended) 5. Modern web browser (Safari, Chrome, Firefox)
Quick Setup:
1. Clone SAM Web repository: git clone https://github.com/SyntheticAutonomicMind/SAM-web.git
2. Start a web server: python3 -m http.server 8000 (from SAM-web directory)
3. Open http://localhost:8000 (same device) or http://YOUR_MAC_IP:8000 (other devices)
4. Enter your API token from SAM Preferences
5. Click Connect to SAM
6. Start chatting
iPad Access: - Chat with SAM while reading on your iPad - Quick questions without switching to your Mac - Touch-optimized interface for tablets
Multi-Device Workflow: - Access SAM from any device on your network - Continue conversations from different locations - Share SAM access with other devices
Remote Work: - Connect from other computers on your network - Use VPN for secure remote access (advanced) - Access SAM while away from your primary Mac
Key Concept: SAM Web is a remote interface to SAM on your Mac. All work happens on your Mac - you're controlling SAM remotely.
| Feature | Native SAM App | SAM Web |
|---|---|---|
| Chat interface | ✅ Full | ✅ Full |
| Conversations | ✅ Full management | ✅ Full management |
| Mini-prompts | ✅ Full management | ✅ Full management |
| Model selection | ✅ All providers | ✅ All configured providers |
| System prompts | ✅ Full selection | ✅ Full selection |
| Personalities | ✅ Full selection | ✅ Full selection |
| Shared topics | ✅ Full management | ✅ Full management |
| Folders | ✅ Organize conversations | ✅ Organize conversations |
| Image generation | ✅ Full (images on Mac) | ✅ Full (images on Mac) |
| File operations | ✅ Full (Mac file system) | ✅ Full (Mac file system) |
| Terminal access | ✅ Full shell access | ✅ Full (Mac terminal) |
| Memory/RAG | ✅ Full (via tools) | ✅ Full (via tools) |
| Document upload | ✅ Drag and drop | ⏳ Planned |
| Voice input/output | ✅ Full support | ⏳ Planned |
| LoRA training | ✅ Complete UI | ❌ Not available |
| Conversation export | ✅ JSON, MD, PDF, TXT | ❌ Not available |
| Memory/RAG UI | ✅ Document management | ❌ Not available |
| Offline mode | ✅ Local models | ❌ Requires network |
Legend: - ✅ = Available - ⏳ = Planned for future release - ❌ = Not available - "(Mac)" = Works via SAM's tools, affects your Mac where SAM runs
Summary: SAM Web provides full chat and configuration access. Features marked "(Mac)" work via SAM's tools but affect your Mac's file system, not the device you're browsing from. Missing: LoRA training UI, conversation export, document management UI.
Architecture: - Pure HTML5, CSS3, JavaScript (no build step required) - Zero dependencies, all assets self-hosted - Server-Sent Events (SSE) for real-time streaming - Bearer token authentication - LocalStorage for settings persistence
Repository: - Source code: github.com/SyntheticAutonomicMind/SAM-web - License: GNU GPL v3.0 (same as SAM) - Contributions welcome
Learn More: See the complete SAM Web Guide for setup instructions, troubleshooting, and detailed feature documentation.
Now that you understand SAM's features, dive deeper into specific topics:
Essential Guides: - Memory & RAG - Master SAM's memory system - Shared Topics - Multi-conversation workflows - Advanced Workflows - Subagents and complex projects
Power User: - Tools Reference - Complete tools reference - Configuration - Complete settings guide - Troubleshooting - Common issues
Developer: - API Reference - REST API documentation - Architecture - System internals
Ready to explore? Start with the feature that interests you most!