ZI_RLM is a standalone FastAPI Recursive Language Model (RLM) sidecar for
OpenWebUI. Its core is a true RLM: the whole corpus is loaded into a PROMPT
variable inside a sandboxed Python REPL, and the model writes code to inspect,
slice and recursively sub-query itself over the data (llm_query/rlm) —
beating context-window limits and enabling whole-corpus reasoning (counting,
exhaustive extraction, cross-referencing) that top-k RAG structurally cannot do.
Around that core ZI_RLM ships a full hybrid RAG retrieval layer (FAISS + BM25) that both feeds the RLM its corpus and powers the faster modes — Deep multi-pass RAG, NMD Compliance Check, per-chat attachment indexing and OCR. So you get one system with a spectrum of modes, from a sub-second lookup to a recursive whole-corpus investigation — all without patching OpenWebUI sources, wired in through a single Function filter, so OpenWebUI and the sidecar upgrade independently.
🇷🇺 Краткое русское резюме — внизу страницы.
Documentation
- 🚀
docs/RUNNING.md— complete step-by-step launch guide (zero → running, in Russian). - 🧠
docs/RAG_vs_RLM.md— plain-language explainer: how RAG and RLM differ and when to use each. - 📘
OPENWEBUI_ZI_RLM.md— full operational documentation.
Native OpenWebUI Knowledge is one-shot top-k RAG: it pre-selects a few chunks and hopes the answer is in them. That structurally fails when a question needs the whole corpus — counting every occurrence, exhaustive extraction, cross-referencing tables — or when retrieval quality on long Russian documents is weak. ZI_RLM is built around the Recursive Language Model to lift that ceiling, and keeps a strong retrieval layer for everything that doesn't need it — without forking OpenWebUI:
- Recursive Language Model at the core.
/rlm/repl(chat command/srlm) implements the RLM paradigm (Zhang et al., MIT CSAIL, arXiv:2512.24601): the corpus lives in aPROMPTvariable inside a sandboxed Python REPL, and the model writes code to grep/slice/aggregate it and recursively sub-queries itself (llm_query/rlm) on the slices. Only code results re-enter context — so it reasons over a corpus far larger than its context window and aggregates across the whole of it, which top-k RAG cannot. Untrusted code runs network-isolated in Docker (default) or rootless bubblewrap, with CPU/memory/PID limits. - Hybrid retrieval layer (feeds the RLM, and serves fast lookups). Dense FAISS (
IndexFlatIP→IndexHNSWFlatauto-switch above 50 000 chunks) fused with SQLite FTS5 BM25 via Reciprocal Rank Fusion. MMR top-K selection, optional cross-encoder rerank, optional LLM query expansion (HyDE / multi-query). This is both thesource=ragcorpus loader for the RLM and the sub-second answer path for ordinary questions. - Deep multi-pass RAG.
/analyzeretrieves a wide candidate set, splits it into batches, asks the chat model to extract facts per batch, then synthesizes a final answer. Streamed via SSE so OpenWebUI shows live progress. The reliable middle ground between a plain lookup and a full RLM run. - Compliance Check.
/compliance/analyzeverifies attached documents against an NMD index with section-level retrieval, JSON matrix, citations and per-file report. Wall-clock-bounded. - Per-chat attachment indexing. Files attached to an OpenWebUI chat are auto-indexed into a per-chat sidecar index
owui_chat_<chat_id>. Native OpenWebUI Knowledge can stay disabled for those models to avoid double-RAG. - Locator-aware citations. Chunks carry source locators (
пункт 3.1,абз. 12,стр. 4,лист/строкаfor spreadsheets, message body and supported attachments for.msg). - Strict static checks. Codebase passes
ruffandmypy(CI-enforced) and is covered by a 177-case pytest suite in development. - Russian/English admin UI with persisted locale, OpenAPI tags/summaries on every endpoint, and a single-file Function filter (
zi_rlm_filter.py) that's also exported aszi_rlm_filter.openwebui.jsonfor direct import.
┌─────────────────────┐ HTTP (X-API-Key) ┌──────────────────────────────┐
│ OpenWebUI │ ─────────────────────► │ ZI_RLM sidecar (FastAPI) │
│ │ │ │
│ Function filter │ ◄───── results ─────── │ /rlm/repl Recursive LM │
│ zi_rlm_filter.py │ │ /analyze Deep RAG │
│ │ ─── multipart upload ─►│ /retrieve /compliance │
└─────────────────────┘ │ /chat-attachments /jobs /ui │
└──────────────┬───────────────┘
│
┌───────────────────────────┼────────────────────────────┐
│ │ │
┌───────▼────────┐ ┌──────────▼──────────┐ ┌─────────▼─────────┐
│ Embeddings │ │ Generation (Ollama │ │ Storage │
│ Ollama or any │ │ or OpenAI-compat.) │ │ FAISS + SQLite │
│ OpenAI-compat. │ │ │ │ FTS5 + uploads │
└────────────────┘ └─────────────────────┘ └───────────────────┘
The Recursive LM executes model-written code in an isolated Docker / bubblewrap sandbox (no network, CPU/memory/PID-limited) — one ephemeral environment per iteration. Details in Recursive Language Model below.
Source layout:
openwebui_zi_rlm/ FastAPI service package
app.py app factory: routers, static UI, OpenAPI metadata
server.py python -m entry, uvicorn bootstrap, public re-exports
config.py pydantic-settings model + env/JSON loader (ZI_RLM_*)
runtime.py shared runtime state, dep-injection, API-key guard
routes/ admin/indexes/documents/jobs/analyze/compliance/chat-attachments
services/ health, jobs, prompting, multi-pass + compliance pipelines
indexing/ extraction, chunking, registry (SQLite), vector_store (FAISS)
ollama_client.py thin async/sync HTTP client with separated timeouts
web/ static admin UI (vanilla JS + i18n: en/ru)
openwebui_functions/
zi_rlm_filter.py OpenWebUI Function (single-file filter)
zi_rlm_filter.openwebui.json importable JSON export, kept in sync
tools/
build_bundle.py release bundler (allowlist-based)
build_filter.py JSON export regenerator
For a full, guided first run (models → sidecar → OpenWebUI filter → indexing → smoke test), follow
docs/RUNNING.md. The summary below is the minimal path.
run_zi_rlm.sh is an idempotent bootstrap: it creates .venv, installs
dependencies, checks the environment (Docker/bwrap/Tesseract/LibreOffice), asks for
host/port, API key, storage dir and LLM endpoints (everything except the path is
optional — Enter keeps the default), then starts the sidecar. Re-running is safe;
answers are remembered.
cd ZI_RLM
./run_zi_rlm.sh # first run: prompts, then starts the server
./run_zi_rlm.sh --check # only verify the environment, do not start
./run_zi_rlm.sh --help # all flags (--reconfigure / --reinstall / --yes)git clone https://github.com/deposist/ZI_RLM.git
cd ZI_RLM
python3 -m venv .venv && source .venv/bin/activate
pip install -r openwebui_zi_rlm_requirements.txt
python3 -m openwebui_zi_rlmThe sidecar binds to 127.0.0.1:8766 by default. Open http://127.0.0.1:8766/ui for the admin panel. To bind a different host/port (e.g. reachable from a Dockerized OpenWebUI), run uvicorn directly: python3 -m uvicorn openwebui_zi_rlm.server:app --host 0.0.0.0 --port 8766.
Each release attaches openwebui_zi_rlm_bundle.zip produced by tools/build_bundle.py. The bundle contains the runtime package, the Function filter, the requirements file and the operations docs — nothing else (no caches, no storage, no tests).
unzip openwebui_zi_rlm_bundle.zip -d /opt/zi_rlm
cd /opt/zi_rlm
python3 -m venv .venv && source .venv/bin/activate
pip install -r openwebui_zi_rlm_requirements.txt
python3 -m openwebui_zi_rlm- Python 3.10+ (CI runs 3.12).
- Tesseract (
tesseract-ocr,tesseract-ocr-rus) — optional CPU OCR fallback. - LibreOffice (
libreoffice) — required to extract legacy.doc,.rtf,.odt. - CUDA toolkit — optional, only for GPU OCR via EasyOCR.
- Docker or bubblewrap (
bwrap) — required only for the Recursive Language Model (/rlm/repl) to sandbox model-generated code. Docker is preferred (docker pull python:3.12-slim, sidecar user in thedockergroup);bwrapis a rootless fallback. With neither available,/rlm/replreturns503while the rest of the sidecar works.
The CI workflow (.github/workflows/ci.yml) installs tesseract-ocr tesseract-ocr-rus libreoffice on Ubuntu and is the canonical reproducible setup.
- Open Admin Panel → Functions → Import, upload
openwebui_functions/zi_rlm_filter.py(or its JSON twin). - Enable the Function in Admin Panel → Functions.
- Make the chat toggle visible:
- turn on Global for
RLM Sidecarto show it for every model; or - attach it only to target models in Workspace → Models → → Filters.
- turn on Global for
- Set valves:
sidecar_url—http://host.docker.internal:8766for Docker-hosted OpenWebUI,http://127.0.0.1:8766for native installs.api_key— required when the sidecar is reachable from Docker / LAN. Mirror it in the sidecar config.sync_sidecar_admin_config— keep on so admins control RAG behavior from the sidecar UI.
- Disable native OpenWebUI Knowledge for the same models. Two RAG layers stacking on the same prompt is the most common cause of "model ignores my context" complaints.
- In the sidecar UI (
http://127.0.0.1:8766/ui→ Настройки), pick the embedding model and a default index.
In OpenWebUI 0.9.2, import success is not enough for the chat menu item to appear. The filter must be active and either global or attached to the selected model.
There are three ways to put data into ZI_RLM:
Drag-and-drop files at http://127.0.0.1:8766/ui. Multi-select supports queueing reindex jobs (В очередь) or forced reindex (Принудительно, cancels active jobs first). Status moves through queued → extracting → vectorizing → indexed; the FAISS vector store is rebuilt once at the end of a batch, not per document.
Add a path under Add path (or POST /indexes/{index_id}/documents/add-path). Paths are accepted only when their resolved root is inside allowed_source_roots; everything else returns 403. Set the allowlist in admin settings or via env (ZI_RLM_ALLOWED_SOURCE_ROOTS=/data/docs,/srv/contracts).
When the user attaches files in a chat, the Function filter reads them from OpenWebUI runtime storage and sends them to /chat-attachments/index. They land in owui_chat_<chat_id> — a per-chat index, separate from your main corpus, scoped by chat id (or session_id/message_id/user_id as fallback). Duplicates are skipped by OpenWebUI file id + content hash; updates soft-delete the previous version.
Supported file types: .txt .md .json .csv .log .htm .html .xml .doc .docx .odt .rtf .pdf .msg .xls .xlsx .xlsm .xlsb. Images and OpenWebUI collections are left for OpenWebUI.
Configuration sources are applied in priority order (higher wins): ZI_RLM_* environment variables → openwebui_zi_rlm_storage/config.json → .env → built-in defaults. So a ZI_RLM_* env var overrides the same key saved in config.json, and config.json overrides .env. Most settings are also editable live from the admin UI, which persists them to config.json; note that an environment variable set for the same key will continue to shadow such edits after a restart.
On first startup ZI_RLM creates config.json with the complete built-in starter
configuration. If an older partial config.json already exists, missing keys
are added while existing values and unknown local keys are kept. Environment
variables still override the file at runtime, but they are not written into the
starter file.
The admin UI has two configuration paths:
- Wizard resets to built-in defaults plus the values entered in the wizard.
If
config.jsonalready exists, the first wizard step shows its path and requires explicit reset confirmation. On Finish the sidecar first copies the currentconfig.jsontoopenwebui_zi_rlm_storage/backups/, then writes the new config as primary. - Настройки exposes the full tabbed settings surface for precise edits without resetting unrelated fields.
Both Wizard and Настройки can load model lists from the currently entered
provider fields before saving. For OpenAI-compatible generation, set provider
to openai, enter a /v1 base URL such as http://127.0.0.1:8081/v1, add the
endpoint API key if needed, and use Обновить модели. Ollama is not required
when embeddings and generation both point to OpenAI-compatible endpoints.
A few of the most common keys (full list in openwebui_zi_rlm/config.py):
| Key | Default | Purpose |
|---|---|---|
storage_dir |
./openwebui_zi_rlm_storage |
Where uploads, FAISS, SQLite live. |
ollama_base_url |
http://127.0.0.1:11434 |
Ollama endpoint when a provider is set to ollama. |
api_key |
empty | Sets X-API-Key requirement for non-health endpoints. |
require_api_key_localhost |
false |
Strict mode for multi-user hosts. |
allowed_source_roots |
[] |
Allowlist for filesystem path indexing. |
embedding_provider |
ollama |
Or openai for an OpenAI-compatible endpoint. |
embedding_model |
empty | Embedding model name. Empty = no embedding work runs. |
deep_generation_provider |
ollama |
Generation provider for Deep RAG, RLM, Compliance and LLM query expansion. Use openai for OpenAI-compatible /v1/chat/completions. |
deep_generation_base_url |
empty | OpenAI-compatible generation API base URL, for example http://127.0.0.1:8081/v1. |
deep_generation_api_key |
empty | API key for the OpenAI-compatible generation endpoint, not the ZI_RLM sidecar key. |
chunk_size / chunk_overlap |
1200 / 120 |
Text chunker budget (characters). |
embedding_batch_size |
16 |
Embedding request batch size. |
embedding_cache_dtype |
fp32 |
fp32 or fp16 (cache only; FAISS stays float32). |
top_k |
8 |
Default retrieval k for /retrieve. |
score_threshold |
0.50 |
Minimum transformed cosine score. |
retrieval_top_k |
70 |
Pre-filter k for the chat filter. |
adaptive_score_margin |
0.20 |
Max distance from best score kept in prompt. |
max_prompt_chunks |
24 |
Hard cap on chunks injected into the chat prompt. |
index_type |
auto |
auto / flat / hnsw. auto switches at hnsw_threshold_chunks. |
hnsw_threshold_chunks |
50000 |
When auto upgrades Flat → HNSW. |
hnsw_m / hnsw_ef_construction / hnsw_ef_search |
32 / 200 / 128 |
HNSW knobs. |
query_expansion_enabled |
false |
LLM-based HyDE / multi-query expansion. |
rerank_enabled |
false |
Cross-encoder rerank via /rerank. |
deep_analysis_enabled |
true |
Auto multi-pass for trigger phrases. |
deep_final_answer |
true |
Sidecar produces the final answer (filter rewrites the assistant message). |
deep_force_all |
false |
Force multi-pass for every question. |
deep_top_k |
70 |
Wide retrieval for deep mode. |
deep_timeout_sec |
900 |
Synchronous deep run cap. |
rlm_repl_enabled |
true |
Master switch for the Recursive Language Model (/rlm/repl). |
rlm_repl_sandbox_backend |
auto |
auto / docker / bwrap. auto prefers Docker, falls back to rootless bubblewrap. |
rlm_repl_sandbox_image |
python:3.12-slim |
Docker image for sandboxed code execution. |
rlm_repl_source |
rag |
Corpus loaded into PROMPT: rag / document / index. |
rlm_repl_max_iterations |
12 |
Max REPL iterations the root model may run. |
rlm_repl_max_recursion_depth |
3 |
Sub-query gate. Effectively boolean today: 0 disables llm_query sub-calls, ≥1 enables one level (sub-models answer flatly and do not recurse further). |
rlm_repl_sub_call_budget |
60 |
Hard cap on total sub-calls per run. |
rlm_repl_timeout_sec |
1200 |
Synchronous recursive-RLM run cap. |
rlm_repl_generation_model |
empty | Recursive-RLM generation model (falls back to deep_generation_model). |
rlm_repl_sandbox_mem_mb / _cpus / _pids |
512 / 1.0 / 128 |
Per-iteration sandbox resource limits. |
compliance_enabled / compliance_auto_enabled |
true / true |
Compliance Check master/auto switches. |
compliance_index_ids |
[] |
NMD index(es) used for /check. |
chat_attachments_enabled |
true |
Per-chat attachment indexing. |
chat_attachment_index_prefix |
owui_chat_ |
Prefix for per-chat indexes. |
enable_ocr |
false |
OCR for image-only PDFs. |
ocr_engine |
easyocr |
Or tesseract for the CPU path. |
ocr_gpu |
true |
EasyOCR on CUDA. |
connect_timeout_sec / request_timeout_sec / stream_idle_timeout_sec |
10.0 / 120.0 / 120.0 |
Separate Ollama HTTP timeouts. |
Every key has a matching ZI_RLM_<UPPER_SNAKE> environment variable. Examples:
ZI_RLM_STORAGE_DIR=/srv/zi-rag
ZI_RLM_API_KEY=$(openssl rand -hex 32)
ZI_RLM_ALLOWED_SOURCE_ROOTS=/data/docs,/srv/contracts
ZI_RLM_OLLAMA_BASE_URL=http://127.0.0.1:11434
ZI_RLM_EMBEDDING_PROVIDER=ollama
ZI_RLM_EMBEDDING_MODEL=bge-m3
ZI_RLM_INDEX_TYPE=auto
ZI_RLM_ENABLE_OCR=true
ZI_RLM_RLM_REPL_SANDBOX_BACKEND=auto
ZI_RLM_RLM_REPL_GENERATION_BASE_URL=http://127.0.0.1:8081/v1
ZI_RLM_RLM_REPL_GENERATION_MODEL=local-llamaZI_RLM exposes a spectrum of modes in chat — from the flagship Recursive Language Model down to a sub-second lookup. With the filter enabled:
| Trigger | Mode | Effect |
|---|---|---|
/srlm <question> |
Recursive LM | Force the RLM: the model writes code in the sandbox and recursively sub-queries itself over the whole corpus. |
Auto-RLM phrases (исследуй, проанализируй, рекурсивно, сколько раз встречается, …) |
Recursive LM | Runs the RLM automatically if rlm_repl_auto_enabled. Takes precedence over /deep. |
/deep <question> |
Deep RAG | Force multi-pass (retrieve → batched facts → synthesis) for this turn. |
Auto-deep phrases (сравни, найди противоречия, все требования, …) |
Deep RAG | Multi-pass /analyze if deep_analysis_enabled. |
/check <question> · /check index:IB <question> |
Compliance | Document-vs-NMD check on the current attachments (optionally naming the NMD index IB). |
| Plain question | Fast RAG | Hybrid retrieval, top-K chunks injected into the prompt — the sub-second path. |
The recursive RLM streams its reasoning (the main model's code, stdout, and each
sub-agent's answer) into an OpenWebUI <think> block; the final answer is shown outside
the reasoning. Trigger phrases are configurable in Настройки.
Heavier is not always better. The RLM's quality rides on the generation model (it writes and verifies its own code). For simple permission/policy ("можно ли X?") lookups,
/deepor/checkare often more reliable than/srlm— seedocs/RAG_vs_RLM.md. Pick the lightest mode that answers the question.
All non-health endpoints require X-API-Key: <api_key> once api_key is set. Public liveness probes hit /health; container probes that need full diagnostics use /health/full (auth-required).
| Method | Path | Purpose |
|---|---|---|
| GET | /health |
Public liveness (no abs paths, no metrics). |
| GET | /health/full |
Full diagnostics + embedding-dimension warnings. |
| GET | /metrics |
Prometheus-style metrics. |
| GET / PUT | /config |
Read or update sidecar config (live). |
| GET | /config/wizard/status |
Check whether the primary config.json exists before running the wizard. |
| POST | /config/wizard |
Save wizard config after backing up the current config.json. |
| GET | /generation/models, /embedding/models |
Discoverable model lists for the selected providers. |
| POST | /generation/models/probe, /embedding/models/probe |
Discover model lists from unsaved provider/base URL/API key fields. |
| GET | /ollama/models |
Legacy alias for /generation/models. |
| POST | /ocr/cache/clear |
Clear OCR result cache. |
| GET / POST / DELETE | /indexes , /indexes/{id} |
Index CRUD. |
| POST | /indexes/{id}/rebuild |
Rebuild FAISS only (reuses chunks). |
| GET | /indexes/{id}/documents |
Paginated document listing. |
| POST | /indexes/{id}/documents/upload |
Single-file multipart upload. |
| POST | /indexes/{id}/documents/upload-batch |
Multipart batch upload. |
| POST | /indexes/{id}/documents/add-path |
Index a local path (allowlist-checked). |
| DELETE / POST | /indexes/{id}/documents/{doc_id} , /delete |
Delete by id or batch. |
| POST | /indexes/{id}/documents/{doc_id}/reindex , /reindex |
Queue reindex (single / batch). |
| POST | /retrieve |
Hybrid retrieval with optional rerank/expansion. |
| POST | /analyze |
Synchronous multi-pass deep RAG. |
| POST | /analyze/jobs |
Async deep job (poll or SSE-stream). |
| GET | /analyze/jobs/{id} |
Job snapshot. |
| GET | /analyze/jobs/{id}/events |
SSE event stream. |
| POST | /analyze/jobs/{id}/cancel |
Cooperative cancel. |
| POST | /rlm/repl |
Synchronous Recursive Language Model (sandboxed code REPL). |
| POST | /rlm/repl/jobs |
Async recursive-RLM job (poll or SSE-stream). |
| GET | /rlm/repl/jobs/{id} |
Recursive-RLM job snapshot. |
| GET | /rlm/repl/jobs/{id}/events |
Recursive-RLM SSE event stream. |
| POST | /rlm/repl/jobs/{id}/cancel |
Cooperative cancel. |
| POST | /compliance/analyze |
Document-vs-NMD verification. |
| POST | /chat-attachments/index |
Index OpenWebUI chat attachments. |
| GET | /jobs, /jobs/{id} |
Indexing job queue. |
| POST | /jobs/{id}/cancel, /indexes/{id}/jobs/cancel |
Cancel index jobs. |
| GET | /ui |
Admin web UI (static). |
| GET | /openapi.json, /docs, /redoc |
FastAPI-provided. |
Quick smoke test once the server is up:
curl -s http://127.0.0.1:8766/health | jq .
curl -s -H "X-API-Key: $ZI_RLM_API_KEY" \
-H "Content-Type: application/json" \
-d '{"query":"что такое КСПД","top_k":5}' \
http://127.0.0.1:8766/retrieve | jq '.results[0]'The Recursive LM (Zhang et al., MIT CSAIL, arXiv:2512.24601) is ZI_RLM's headline mode and a fundamentally different paradigm from RAG. Instead of pre-selecting a few chunks and hoping, it gives the model programmatic control over the whole corpus:
- Corpus → variable. The selected corpus (
source=ragtop-k,source=documentfull text, orsource=index) is loaded into aPROMPTvariable inside a sandboxed Python REPL. The root model sees only its length plus a head/tail preview. - Model writes code. Each iteration the model emits one
pythonblock that greps/slices/aggregatesPROMPTandprint()s intermediate results. Variables persist across iterations; common stdlib modules (re,json,collections, …) are pre-imported. - Recursive sub-queries. Code may call
llm_query(text, question)/rlm(...)to ask a fresh sub-model about a slice. The sandbox has no network, so the host fulfils these via a probe→resolve two-pass protocol (depth- and budget-limited). - Finish. Setting
FINAL_ANSWERends the loop; otherwise the loop runs up torlm_repl_max_iterationsthen synthesizes from collected state.
The whole run streams into an OpenWebUI <think> block (the model's code, stdout, and each
sub-agent's answer); the final answer is shown outside the reasoning.
Sandboxing. Untrusted code runs network-isolated, read-only-root, non-root, with
CPU/memory/PID limits, in Docker (one ephemeral --network none container per
iteration) or rootless bubblewrap. In-sandbox imports are restricted to a safe
standard-library allowlist as a second defense layer. Backend selection:
rlm_repl_sandbox_backend = auto | docker | bwrap. With neither backend available,
/rlm/repl returns 503 and every other mode keeps working.
When to use it. Tasks that need completeness or whole-corpus reasoning — counting occurrences across a document, exhaustive extraction, cross-references — where top-k RAG structurally misses information. For ordinary lookups the lighter modes below are faster and cheaper.
🧠 RAG vs RLM, in plain language:
docs/RAG_vs_RLM.mdexplains the difference without jargon and why the RLM depends heavily on the generation model. Answer correctness is bounded by the chat model, not by retrieval — for permission/policy ("можно ли X?") questions,/deepor/checkare more reliable than/srlm, and a capable instruction-following model matters more than the mode.
The hybrid retrieval layer underpins every mode — it loads the RLM's PROMPT
(source=rag), supplies Deep RAG's candidate set, and answers plain questions directly.
- Chunking. Documents are split with structure-aware boundaries (headings, paragraph breaks, numbered clauses) targeting
chunk_sizechars,chunk_overlapchars overlap. - Indexing. Embeddings are written to FAISS (
FlatorHNSW); the same chunks are mirrored into SQLite FTS5 for BM25. - Query expansion (optional). LLM produces HyDE / multi-query variants; user-defined
query_synonymsadd domain-specific aliases (КСПД/ТСПД bundled). - Hybrid search. Dense top-K (
retrieval_top_k) + BM25 top-K, fused via Reciprocal Rank Fusion. BM25-only hits getscore = 1 / (1 + max(0, fts_rank − 1))— no hardcoded 0.72 plateau. - Filter. Score floor (
score_threshold), adaptive margin, query-term-hit lexical filter, MMR diversification. - Optional rerank. Cross-encoder via OpenAI-compatible
/rerankendpoint. - Prompt packing.
max_prompt_chunksinjected as full text in the first batch; the rest go into compact<details>Источники</details>lines so OpenWebUI doesn't truncate the prompt.
For deep mode the same retrieval runs with deep_top_k, then chunks are split into deep_max_batches × deep_batch_chars batches; each batch produces structured facts; the final pass synthesizes the answer.
Deep RAG (/analyze, also auto-triggered for analytical phrases): wide retrieval → batched fact extraction → final synthesis. Streams progress via SSE — the OpenWebUI filter shows live status (анализ пачки 3/8, финальный синтез, …).
Compliance Check (/compliance/analyze, command /check): extracts attached files (.doc .docx .xls .xlsx .pdf .msg + text-like) into a temp dir, splits into sections (compliance_section_chars, capped by compliance_max_sections), retrieves NMD requirements per section, asks the model to produce findings, then renders a JSON matrix + per-file report. Temporary checked files are deleted after analysis and never added to permanent indexes (unless chat-attachment indexing is on, in which case they're also saved to the per-chat index for follow-ups).
Both modes need a generation model. There is no production fallback: pick one in admin UI, pass generation_model in the request, or set ZI_RLM_DEEP_GENERATION_MODEL / ZI_RLM_COMPLIANCE_GENERATION_MODEL. If the model is missing from /api/tags, the sidecar returns 409 with the available list (or 502 if /api/tags itself is down).
PDFs with a text layer are read directly. Image-only pages use OCR when enable_ocr=true. Recommended GPU defaults:
ocr_engine = easyocr
ocr_gpu = true
ocr_languages = rus+eng
ocr_model_storage_dir = <storage_dir>/easyocr_modelsPass ZI_RLM_GPU to bind a specific CUDA device. Set ocr_engine=tesseract only when you explicitly want the CPU path. After enabling OCR, reindex existing image-only PDFs.
openwebui_zi_rlm_storage/
config.json persisted admin settings
registry.sqlite documents, jobs, FTS5
uploads/ raw uploaded files
indexes/<index_id>/ vectors.faiss + vector_map.json + metadata
easyocr_models/ downloaded EasyOCR weights (when OCR is on)
Move with ZI_RLM_STORAGE_DIR=/some/where. The whole openwebui_zi_rlm_storage/ tree is git-ignored — never commit it.
In-process FAISS cache: LRU of size 32. Indexes are evicted when deleted, replaced when rebuilt, and cleared whenever the live config refreshes (storage paths might have changed). To clear manually, restart the sidecar.
ruff check openwebui_zi_rlm/ openwebui_functions/ tools/
mypy openwebui_zi_rlm openwebui_functions
python3 tools/build_bundle.py --output dist/openwebui_zi_rlm_bundle.zipCI runs the same lint/type/build gates. Releases are cut by pushing a tag:
git tag v1.0.0
git push --tags.github/workflows/release.yml rebuilds openwebui_zi_rlm_bundle.zip, attaches it to a GitHub Release, and uses RELEASE_NOTES_<version>.md as the body when present.
| Symptom | Likely cause | Fix |
|---|---|---|
/retrieve returns empty results even for known content |
Embedding model not set, or its dimension mismatches existing indexes | Set embedding_model; check /health/full embedding_model_dimension.warnings; force-reindex affected indexes. |
| Sidecar 401/403 from OpenWebUI Function | api_key mismatch between filter valves and sidecar config |
Sync api_key in both places. Restart the filter from OpenWebUI Functions panel. |
/analyze or /compliance/analyze returns 409 |
Generation model not selected or missing from the selected generation provider | Pick a model in admin UI. For Ollama, pull it with ollama pull <model>; for OpenAI-compatible, verify /v1/models. |
add-path returns 403 |
Path is outside allowed_source_roots |
Add the parent directory to the allowlist. |
| Deep job 410 after sidecar restart | Deep jobs are in-memory only | OpenWebUI filter falls back to synchronous /analyze automatically. |
| OpenWebUI shows the same context twice | Native Knowledge still enabled for that model | Disable native Knowledge for models that use the ZI_RLM filter. |
| Image-only PDFs return empty text | enable_ocr=false |
Enable OCR, ensure Tesseract or EasyOCR is installed, reindex. |
- Python 3.10+ (CI: 3.12).
- OpenWebUI with the Function API (0.9.0+).
- Ollama or OpenAI-compatible endpoints for embeddings and generation (Giga, llama.cpp, OpenAI proper).
- FAISS-CPU. GPU OCR via EasyOCR + CUDA optional; Tesseract supported on CPU.
See LICENSE.
ZI_RLM — внешний RLM-сайдкар для OpenWebUI. Его ядро — настоящий Recursive Language Model (/rlm/repl, команда /srlm): весь корпус кладётся в переменную PROMPT в песочнице с Python, и модель сама пишет код, чтобы его исследовать, и рекурсивно опрашивает себя по кускам (llm_query). Это даёт рассуждение по всему корпусу (подсчёты, сплошная выборка, сверки таблиц), чего top-k RAG не умеет. Вокруг ядра — полноценный слой гибридного поиска FAISS+BM25, который и грузит корпус в RLM, и обслуживает быстрые режимы: multi-pass deep-анализ (/deep), проверка документов на НМД (/check), индексация вложений чата в отдельный per-chat индекс. Не патчит исходники OpenWebUI: всё подключается через одну Function (openwebui_functions/zi_rlm_filter.py). Код RLM исполняется изолированно в Docker (или rootless bwrap), без сети; рассуждения суб-агентов и основной модели — в блоке <think>, итог отдельно. UI на двух языках, OCR на GPU/CPU, поддержка .doc/.docx/.xls/.xlsx/.pdf/.msg/.rtf/.odt/.txt/.md и т.п.
Важно: качество ответа упирается в генеративную модель. На слабой модели для вопросов «можно ли X?» надёжнее
/deep//check, чем/srlm— см.docs/RAG_vs_RLM.md.
Запуск:
pip install -r openwebui_zi_rlm_requirements.txt
python3 -m openwebui_zi_rlmАдминка: http://127.0.0.1:8766/ui · Health: http://127.0.0.1:8766/health · OpenAPI: http://127.0.0.1:8766/docs
Пошаговый запуск: docs/RUNNING.md · Чем RAG отличается от RLM: docs/RAG_vs_RLM.md · Полная операционная документация: OPENWEBUI_ZI_RLM.md.