docs: add comprehensive documentation with README and detailed guides

- Add user-friendly README.md with quick start guide - Create docs/ folder with structured technical documentation: - installation.md: Build and setup instructions - configuration.md: Complete config reference - usage.md: CLI usage guide with examples - architecture.md: System design and patterns - components/: Deep dive into each component (OpenQueryApp, SearchTool, Services, Models) - api/: CLI reference, environment variables, programmatic API - troubleshooting.md: Common issues and solutions - performance.md: Latency, throughput, and optimization - All documentation fully cross-referenced with internal links - Covers project overview, architecture, components, APIs, and support See individual files for complete documentation.
2026-03-19 10:01:58 +01:00
parent b28d8998f7
commit 65ca2401ae
16 changed files with 7073 additions and 0 deletions
--- a/docs/usage.md
+++ b/docs/usage.md
@@ -0,0 +1,483 @@
+# Usage Guide
+
+Complete reference for using the OpenQuery command-line interface.
+
+## 📋 Table of Contents
+
+1. [Basic Usage](#basic-usage)
+2. [Command Reference](#command-reference)
+3. [Examples](#examples)
+4. [Output Format](#output-format)
+5. [Tips and Tricks](#tips-and-tricks)
+
+## Basic Usage
+
+### Simplest Form
+```bash
+openquery "your question here"
+```
+
+That's it! OpenQuery will:
+1. Generate search queries
+2. Search the web
+3. Extract relevant content
+4. Stream an answer with sources
+
+### Common Pattern
+```bash
+openquery [OPTIONS] "your question"
+```
+
+Quotes around the question are recommended to preserve spaces.
+
+## Command Reference
+
+### Main Command
+
+#### `openquery [options] <question>`
+
+Ask a question and get an AI-powered answer with citations.
+
+**Arguments**:
+- `question` (positional, one or more words) - The question to ask
+
+**Options**:
+
+| Option | Aliases | Type | Default | Description |
+|--------|---------|------|---------|-------------|
+| `--chunks` | `-c` | int | 3 (from config) | Number of top relevant content chunks to include in context |
+| `--results` | `-r` | int | 5 (from config) | Number of search results to fetch per generated query |
+| `--queries` | `-q` | int | 3 (from config) | Number of search queries to generate from your question |
+| `--short` | `-s` | bool | false | Request a concise, to-the-point answer |
+| `--long` | `-l` | bool | false | Request a detailed, comprehensive answer |
+| `--verbose` | `-v` | bool | false | Show detailed progress information and debug output |
+
+**Behavior**:
+- Short and long are mutually exclusive but can both be omitted (balanced answer)
+- If both `--short` and `--long` are specified, `--long` takes precedence
+- Options override configuration file defaults
+
+#### `openquery configure [options]`
+
+Configure OpenQuery settings (API key, model, defaults).
+
+**Options**:
+
+| Option | Type | Description |
+|--------|------|-------------|
+| `--interactive` / `-i` | bool | Launch interactive configuration wizard |
+| `--key` | string | Set the OpenRouter API key |
+| `--model` | string | Set the default model |
+| `--queries` | int? | Set default number of queries |
+| `--chunks` | int? | Set default number of chunks |
+| `--results` | int? | Set default number of results |
+
+**Examples**:
+```bash
+# Interactive wizard
+openquery configure -i
+
+# Set just the API key
+openquery configure --key "sk-or-..."
+
+# Set multiple defaults non-interactively
+openquery configure --model "deepseek/deepseek-v3.2" --queries 5 --chunks 4
+```
+
+**Note**: Options with `?` are nullable; only provided values are updated.
+
+## Examples
+
+### Everyday Queries
+
+**Simple factual question**:
+```bash
+openquery "What is the speed of light?"
+```
+
+**Multi-word question**:
+```bash
+openquery "How do solar panels work?"
+```
+
+**Question with special characters**:
+```bash
+openquery "What's the weather in New York?"
+```
+
+### Customizing Output
+
+**Get a quick answer**:
+```bash
+openquery -s "Who is the CEO of Tesla?"
+```
+Output: "Elon Musk is the CEO of Tesla." (minimal explanation)
+
+**Get detailed analysis**:
+```bash
+openquery -l "Explain how nuclear fusion works"
+```
+Output: Multi-paragraph detailed explanation with scientific details
+
+**See everything**:
+```bash
+openquery -v "What is machine learning?"
+```
+Output: Shows all progress messages alongside the answer
+
+### Adjusting Search Depth
+
+**Minimal search** (fast, cheap):
+```bash
+openquery -q 1 -r 2 -c 1 "What time is it in London?"
+```
+- 1 generated query
+- 2 results per query
+- 1 context chunk
+
+**Thorough research** (slow, comprehensive):
+```bash
+openquery -q 8 -r 15 -c 5 "History and applications of cryptography"
+```
+- 8 diverse queries
+- 15 results each
+- 5 top chunks
+
+**Balanced (recommended defaults)**:
+```bash
+openquery "Latest advancements in CRISPR technology"
+```
+- 3 queries
+- 5 results each
+- 3 top chunks
+
+### Combining Options
+
+**Verbose custom search**:
+```bash
+openquery -v -q 5 -r 10 -c 4 "What are the ethical implications of AI?"
+```
+
+**Short answer with more context**:
+```bash
+openquery -s -c 5 "Python vs JavaScript for web development"
+```
+
+**Long answer, lots of research**:
+```bash
+openquery -l -q 10 -r 20 -c 6 "Complete guide to quantum computing"
+```
+
+### Practical Use Cases
+
+**News and Current Events**:
+```bash
+openquery "Latest developments in the Ukraine conflict"
+```
+
+**Technical Questions**:
+```bash
+openquery "How to set up a PostgreSQL replication cluster"
+```
+
+**Health Information** (verify with doctor!):
+```bash
+openquery "What are the symptoms of vitamin D deficiency?"
+```
+
+**Cooking**:
+```bash
+openquery "How to make authentic Italian pizza dough"
+```
+
+**Travel**:
+```bash
+openquery "Best things to do in Tokyo in spring"
+```
+
+**Programming**:
+```bash
+openquery "Rust vs Go for backend development in 2025"
+```
+
+### Configuration Examples
+
+**Set up for the first time**:
+```bash
+openquery configure -i
+# Follow prompts to enter API key, choose model, set defaults
+```
+
+**Switch to a different model**:
+```bash
+openquery configure --model "google/gemini-3-flash-preview"
+```
+
+**Update default number of queries**:
+```bash
+openquery configure --queries 5
+```
+
+**Set cost-effective defaults**:
+```bash
+openquery configure --model "qwen/qwen3.5-flash-02-23" --queries 2 --chunks 2 --results 3
+```
+
+**Check your configuration**:
+```bash
+cat ~/.config/openquery/config
+```
+
+## Output Format
+
+### Standard Output (Streaming)
+
+The answer streams in real-time, character by character, like this:
+
+```
+⠋ Generating search queries...  (spinner with status)
+⠹ Searching web...
+⠸ Fetching articles...
+⠼ Processing embeddings...
+⠴ Generating answer...
+Assistant: Quantum entanglement is a phenomenon where pairs or groups of
+particles interact in ways such that the quantum state of each particle
+cannot be described independently of the others, even when separated by
+large distances.
+
+[Source 1: Understanding Quantum Mechanics](https://example.com/quantum)
+[Source 2: Quantum Physics Overview](https://example.com/physics)
+```
+
+### Verbose Mode Output (`-v`)
+
+When `--verbose` is enabled, you see detailed progress:
+
+```
+[Generating 3 search queries based on your question...]
+[Generated queries: 
+  1. quantum entanglement definition
+  2. how quantum entanglement works
+  3. quantum entanglement Bell's theorem
+]
+[Searching web for 'quantum entanglement definition'...]
+[Searching web for 'how quantum entanglement works'...]
+[Searching web for 'quantum entanglement Bell's theorem'...]
+[Fetched 15 search results total]
+[Fetching article 1/12: physicsworld.com]
+[Fetching article 2/12: nature.com]
+...
+[Fetching article 12/12: scientificamerican.com]
+[Extracted 48 content chunks]
+[Generating embeddings: batch 1/4]
+[Generating embeddings: batch 2/4]
+[Generating embeddings: batch 3/4]
+[Generating embeddings: batch 4/4]
+[Ranked chunks by relevance]
+[Found top 3 most relevant chunks overall. Generating answer...]
+
+Assistant: Quantum entanglement is a fundamental phenomenon in quantum
+mechanics where...
+```
+
+### Source Citations
+
+Sources are formatted as markdown links in the answer:
+```
+[Source 1: Article Title](https://example.com/article)
+```
+
+These appear inline where the AI references that source. Multiple sources can be cited in a single paragraph.
+
+### Error Output
+
+Errors are written to stderr and exit with non-zero status:
+
+```
+[Error] API Key is missing. Set OPENROUTER_API_KEY environment variable or run 'configure -i'.
+```
+
+## Tips and Tricks
+
+### Speed Tips
+
+1. **Reduce concurrency limits** (edit `SearchTool.cs` if constantly rate-limited)
+2. **Reduce `--results`** - fewer articles to fetch and process
+3. **Reduce `--queries`** - fewer parallel searches
+4. **Use local SearxNG** - minimize network latency to search backend
+5. **Cache results** - future enhancement could add caching
+
+### Quality Tips
+
+1. **Increase `--chunks`** to 4-5 for complex topics
+2. **Increase `--queries`** to 5-8 for broad exploration
+3. **Use `--long`** for deep topics that need elaboration
+4. **Check `-v` output** to see which sources were selected
+5. **Try different models** - some are better at synthesis, others at facts
+
+### Cost Tips
+
+1. **Use `qwen/qwen3.5-flash-02-23`** - cheapest good model
+2. **Reduce `--chunks`** and `--results`** - fewer tokens in context
+3. **Use `--short`** - shorter answers use fewer completion tokens
+4. **Monitor usage** at [openrouter.ai](https://openrouter.ai) dashboard
+
+### Workflow Tips
+
+**Iterative deepening**:
+```bash
+# Start broad
+openquery -v "machine learning"
+
+# Identify subtopics from answer, then dive deeper
+openquery "What is transformer architecture in LLMs?"
+```
+
+**Compare answers**:
+```bash
+# Same question with different models
+OPENROUTER_MODEL="qwen/qwen3.5-flash-02-23" openquery "question"
+OPENROUTER_MODEL="google/gemini-3-flash-preview" openquery "question"
+```
+
+**Save answers**:
+```bash
+openquery "What is Docker?" > answer.md
+# answer.md will contain the streamed output (including spinner chars, so filter):
+openquery "What is Docker?" 2>/dev/null | sed 's/.\x08//g' > clean-answer.md
+```
+
+### Shell Aliases and Functions
+
+Add to `~/.bashrc` or `~/.zshrc`:
+
+```bash
+# Short alias
+alias oq='openquery'
+
+# With common options
+alias oql='openquery -l -q 5 -r 10'  # long, thorough
+alias oqs='openquery -s'              # short
+alias oqv='openquery -v'              # verbose
+
+# Function to save output cleanly
+oqsave() {
+    openquery "$@" 2>/dev/null | sed 's/.\x08//g' > "answer-$(date +%Y%m%d-%H%M%S).md"
+}
+```
+
+### Scripting
+
+```bash
+#!/bin/bash
+# batch-questions.sh
+
+while IFS= read -r question; do
+    echo "## $question" >> research.md
+    echo "" >> research.md
+    openquery -l "$question" 2>/dev/null | sed 's/.\x08//g' >> research.md
+    echo "" >> research.md
+done < questions.txt
+```
+
+### Chaining with Other Tools
+
+Pipe to `jq` (if you modify to output JSON):
+```bash
+# Future: openquery --json "question" | jq '.answer'
+```
+
+Pipe to `pbcopy` (macOS) or `xclip` (Linux):
+```bash
+openquery "quick fact" 2>/dev/null | sed 's/.\x08//g' | pbcopy
+```
+
+Filter sources:
+```bash
+openquery "topic" 2>/dev/null | sed 's/.\x08//g' | grep -E '^\[Source'
+```
+
+## Keyboard Interrupts
+
+- **Ctrl+C** during processing: Cancels current operation, exits gracefully
+- **Ctrl+C** during streaming answer: Stops streaming, shows partial answer
+- **Ctrl+Z** (suspend): Not recommended; may leave background tasks running
+
+OpenQuery uses proper cancellation tokens to clean up resources on interrupt.
+
+## Exit Codes
+
+| Code | Meaning |
+|------|---------|
+| 0 | Success - answer was generated |
+| 1 | Error - see stderr message |
+| 2 | Configuration error (missing API key) |
+
+You can check the exit code in shell scripts:
+```bash
+openquery "question"
+if [ $? -eq 0 ]; then
+    echo "Success!"
+else
+    echo "Failed"
+fi
+```
+
+## Limitation and Workarounds
+
+### Question Length
+Very long questions (>2000 chars) may be truncated by the LLM's context window or cause token limits.
+
+**Workaround**: Keep questions concise; discuss complex multi-part questions separately.
+
+### Answer Length Limits
+The LLM may hit `max_tokens` limits for very complex questions.
+
+**Workaround**: Use `--long` flag (already maximizes allowed tokens) or break into sub-questions.
+
+### Rate Limiting
+OpenRouter may rate limit if you send too many requests too quickly.
+
+**Symptoms**: 429 errors, occasional timeouts.
+
+**Workaround**: The built-in retry (Polly) handles this automatically. For persistent issues:
+- Reduce concurrency (edit code)
+- Add delays between queries
+- Upgrade OpenRouter plan
+
+### SearxNG Timeouts
+Large SearxNG responses or slow targets may timeout.
+
+**Workaround**: Reduce `--results` or check SearxNG logs. Nothing to do on OpenQuery side (HTTP client timeout is ~30s default).
+
+---
+
+## Next Steps
+
+- [Architecture](architecture.md) - Understand how OpenQuery works under the hood
+- [Configuration](configuration.md) - Fine-tune your setup
+- [Troubleshooting](troubleshooting.md) - Solve common problems
+
+---
+
+**Quick Reference Card**
+
+```bash
+# Basic
+openquery "question"
+
+# Quick fact
+openquery -s "question"
+
+# Deep research
+openquery -l -q 5 -r 10 -c 4 "question"
+
+# See progress
+openquery -v "question"
+
+# Configure
+openquery configure -i
+
+# Check config
+cat ~/.config/openquery/config
+```