AI Generation
Answer stream (RAG)
Stream a RAG answer from the local LLM as Server-Sent Events.
POST
Overview
Same retrieval and prompting asPOST /answer, but the LLM response is streamed as Server-Sent Events (SSE) instead of a single JSON body.
When no passages are retrieved, the server does not call the LLM: you get one token event with I don’t have enough information to answer that question. then done with context_count: 0.
Typical event order:
meta- model, sources, and context count (sent once)token- one or more answer text deltasdone- full answer echo and metadataerror- if the LLM call fails (instead ofdone)
Requires Ollama running on the host. Start the stack with
moorcheh-edge up (use --skip-ollama for search-only).Request body
Same fields as Answer (RAG):string
required
Original question text (included in the LLM prompt and echoed in the response).
array
required
JSON array of floats used for similarity search. Length must match the store dimension (384 for text stores).
number
default:"5"
Number of passages to retrieve for context. Capped at 100.
number
default:"0"
Minimum search score when
kiosk_mode is true.boolean
default:"false"
When
true, filters retrieved passages below threshold.string
Optional system instruction (replaces the default RAG system prompt).
Optional instruction appended before the user question in the final user message.
array
Prior turns:
[{"role": "user"|"assistant", "content": "..."}].number
default:"0.2"
LLM sampling temperature (0.0–2.0).
SSE events
The
sources array in meta matches the shape returned by POST /search (id, score, label, text).
Errors
Non-streaming HTTP errors (empty store, invalid vector, LLM not configured) return JSON with a4xx status before SSE starts. Once streaming begins, LLM failures arrive as an SSE error event.
Related
- Answer (RAG)
- Python: answer_stream()
- Voice server: POST /ask/stream - proxies this endpoint on edge hardware